We have just released several new versions of the Ping Identity Directory Server to address a security issue that we discovered. The issue is in a component of the server that is only enabled when setting up the Delegated Admin product, and customers who are using that product are strongly advised to upgrade. Customers who are not using the Delegated Admin product should not be affected by the issue.

The following new versions are now available and contain the fix for this issue:

• 9.3.0.1
• 9.2.0.2
• 9.1.0.3
• 8.3.0.9

The security issue was discovered internally, and we have no reason to believe that it has been independently discovered or exploited. Ping is not currently prepared to provide additional information about the vulnerability at this time, but is expected to release a security advisory with additional details in the future.

We have just released version 9.3.0.0 of the Ping Identity Directory Server. See the release notes for a complete overview of changes, but here’s my summary:

Summary of New Features and Enhancements

• Added support for data encryption restrictions [more information]
• Added the ability to freeze the encryption settings database [more information]
• Added the ability to set up the server with a pre-existing encryption settings database [more information]
• Added support for monitoring the availability of the encryption settings database [more information]
• Added other data encryption improvements [more information]
• Added an aggregate pass-through authentication handler [more information]
• Added a PingOne pass-through authentication handler [more information]
• Improved dsreplication performance in topologies with a large number of servers and/or high network latency between some of the servers
• Added more options for allowing pre-encoded passwords [more information]
• Added the ability to use the proxied authorization v1 or v2 request control in password modify extended requests
• Updated the Directory REST API to provide support for the password modify, get password quality requirements, and suggest password extended operation types [more information]
• Added a disallowed characters password validator [more information]
• Added a UTF-8 password validator [more information]
• Added the ability to include ds-pwp-modifiable-state-json in add operations [more information]
• Added the ability to automatically apply changes to TLS protocol and cipher suite configuration [more information]
• Added new account-authenticated and account-deleted account status notification types [more information]
• Added configuration properties for managing the configuration archive [more information]
• Added a new replication-missing-changes-risk alert type [more information]
• Added a new replication-not-purging-obsolete-replicas alert type [more information]
• Added a new check-replication-domains tool that can list known replication domains identify any that may be obsolete
• Added a --showPartialBacklog argument to dsreplication status
• Added the ability to synchronize Boolean-valued attributes to the PingOne sync destination
• Updated replace-certificate to support obtaining new certificate information from PEM files
• Added support for encrypted PKCS #8 private keys [more information]
• Added caching support to the PKCS #11 key manager provider [more information]
• Added the ability to specify the start and end times for the range of log messages to include in collect-support-data archives when invoking the tool as an administrative task

Summary of Bug Fixes

• Fixed an issue when modifying ds-pwp-modifiable-state-json with other attributes [more information]
• Fixed an issue that could prevent the server from properly building indexes with very long names
• Fixed an issue that could cause the server to omit matching entries when configuring compact-common-parent-dn values [more information]
• Fixed an issue in which failover may not work properly after updating a Synchronization Server instance with manage-profile replace-profile
• Fixed an issue with replace modifications for attributes containing variants with options [more information]
• Improved support for passwords containing characters with multiple encodings [more information]
• Fixed an issue that could prevent obsolete replicas from being automatically purged in certain circumstances
• Fixed an issue that could prevent the servers in a replication topology from being able to select the authoritative server for maintaining information in the topology registry
• Increased timeouts used by the dsreplication tool to reduce the chance that they would be incorrectly encountered when interacting with a large replication topology
• Fixed an issue that caused the Directory REST API to always include the permissive modify request control when updating entries
• Improved access control behavior for the password policy state extended operation [more information]
• Fixed an issue in which subtree searches based at the server’s root DSE could omit entries from backends with base DNs subordinate to those of other backends
• Fixed an issue that could prevent a user from using grace logins to change their own password in a modify request that contained the proxied authorization request control
• Fixed an issue with substring filters containing logically empty substrings [more information]
• Improved error handling when using automatic authentication with client certificates [more information]
• Improved Directory Proxy Server error handling when using the rebind authorization method [more information]
• Fixed an issue that prevented including permit-export-reversible-passwords privilege in the default set of root privileges
• Fixed an issue that could cause manage-profile setup to complain about being unable to find certain utilities used by the collect-support-data tool
• Fixed an error that could occur if an archived configuration file was removed in the middle of an attempt to back up the config backend
• Fixed an issue that prevented the Directory Proxy Server from logging search result entry messages for entries passed through from a backend server
• Fixed an issue when synchronizing account state from Active Directory when using modifies-as-creates
• Suppressed servlet information in HTTP error messages by default
• Restricted the RSA key size for inter-server certificates to a maximum of 3072 bits
• Fixed an issue with base DN case sensitivity when enabling replication with a static topology
• Changed the result code used when rejecting an attempt to change a password that is within the minimum age from 49 (invalidCredentials) to 53 (unwillingToPerform)
• Fixed an issue that could cause the server to return multiple password validation details response controls in the response to a password modify extended request
• Fixed an issue that could prevent the server from returning a generated password for a password modify extended operation processed with the no-operation request control

Encryption Settings Database Improvements

We have made a set of changes to the way that the server manages and interacts with the encryption settings database. When used in combination, this can allow for a separation of duties between those responsible for managing the Directory Server itself and those responsible for managing data encryption, which could be used to limit the access that server administrators have to encrypted data. However, even in environments where this strict separation of duties is not required, these changes can still substantially improve the overall security of the directory environment.

These changes come in the form of four key enhancements:

• We have introduced the ability to impose restrictions on the ways that administrators can interact with encrypted data. These restrictions are defined in the encryption settings database itself and can prevent administrators from doing any or all of the following:
  • Disabling data encryption
  • Changing the cipher stream provider used to protect the encryption settings database
  • Exporting the encryption settings definitions
  • Exporting or backing up backend data in unencrypted form
  • Exporting or backing up backend data in a form that is encrypted with a supplied passphrase rather than an encryption settings definition
  • Using the encrypt-file tool to decrypt files

• We have added the ability to freeze the encryption settings database with a passphrase. If the encryption settings database has been frozen, then no changes may be made to it, including creating, importing, or removing definitions, changing which is the preferred definition, and altering the set of data encryption restrictions. If any changes are needed, then the encryption settings database may be un-frozen using the same passphrase that was initially used to freeze it.

• We have added the ability to set up the server with a pre-existing encryption settings database. This database should already have the desired set of definitions, and it may be configured with data encryption restrictions and frozen so that no changes will be allowed. This database will also be tied to a specific cipher stream provider used to protect its contents. To set up the server with a pre-existing encryption settings database, you should use manage-profile setup with a server profile that has the following characteristics:
  • The setup-arguments.txt file must contain the new --encryptDataWithPreExistingEncryptionSettingsDatabase argument.
  • The configuration changes needed to set up the associated cipher stream provider must be included in dsconfig batch files contained in the pre-setup-dsconfig directory.
  • The encryption settings database itself, along with any metadata files that the cipher stream provider might need, should be included in the appropriate locations below the server-root/pre-setup directory.

• We have added support for a new monitor provider that can periodically ensure that the server can read the contents of the encryption settings database without relying on any caching that the cipher stream provider may normally use to improve performance and reliability. Not only does this offer better overall monitoring for the health of the server, but it can also be used to take action if the information that it needs to interact with the encryption settings database has been disabled. For example, if the cipher stream provider relies on an external key or secret (e.g., from Amazon Key Management Service, Amazon Secrets Manager, Azure Key Vault, CyberArk Conjur, or HashiCorp Vault) to be able to unlock the encryption settings database, and that key or secret has been intentionally removed or revoked, then it may be desirable to take action to limit the server’s ability to access encrypted data, like entering lockdown mode or shutting down entirely.

When all of these are combined, and the encryption settings database is protected by a cipher stream provider that relies on an external service, the server administrators could be able to maintain the server with data encryption enabled, but with substantially restricted access to the data that it contains. Although it’s not yet available as an option, this could be useful in environments like PingOne Advanced Services, where Ping personnel can manage a Directory Server deployment on behalf of an organization with limited access to that organization’s data.

Other Data Encryption Improvements

We have made a number of other additional improvements in the server’s support for data encryption. These include:

• We added a new --key-factory-iteration-count argument to the encryption-settings create command to make it possible to specify the PBKDF2 iteration count to use for the new definition. When creating a new encryption settings definition that is backed by a randomly generated secret, the server will now default to using an iteration count of 600,000 in accordance with the latest OWASP guidelines.

• We updated most cipher stream providers to make it possible to explicitly specify the PBKDF2 iteration count that they use when deriving the key used to protect the encryption settings database. They also now use a higher default iteration count of 600,000 when creating a new database.

• We updated the file-based cipher stream provider to support using a separate metadata file with additional details about the encryption that it uses to protect the encryption settings database. When setting up a new instance of the server in a manner that uses this cipher stream provider, a metadata file will be automatically generated to allow it to use stronger encryption to protect the database than has been used in the past.

• We have improved the strength of the encryption used to protect encryption settings exports, backups, LDIF exports, encrypted log files, and other types of file encryption. We now prefer 256-bit AES over 128-bit AES when it’s available, and we use a higher PBKDF2 iteration count to protect the key.

• We have improved the performance of file encryption and decryption operations performed by the server in the common case in which the encryption uses an encryption settings definition rather than a separate passphrase. Although the server (or standalone tools that need to access encryption settings definitions) may take a little longer to start up with the stronger encryption settings that are in place, the use of caching should dramatically reduce the cost of subsequent encryption and decryption operations.

• We updated the encryption settings backend to provide additional information about the definitions contained in the encryption settings database. The base entry for the backend will also indicate whether the encryption settings database is frozen and/or configured with any data encryption restrictions.

• We updated setup so that if it is configured to generate a tools.pin file with the default password that command-line tools may use to authenticate to the server, that password will now be automatically encrypted if data encryption is enabled.

New Pass-Through Authentication Handlers

The Directory Server has long supported the ability to use pass-through authentication, in which a bind attempt against a local entry may ultimately be forwarded to an external service to verify the credentials, optionally setting those credentials if they are confirmed to be correct.

Initially, pass-through authentication was only supported for other LDAP servers. Later, we introduced support for passing through authentication attempts to PingOne. In the 9.0 release, we introduced a new pluggable pass-through authentication plugin that made it possible to use the Server SDK to develop custom authentication handlers that could be used to target other types of services.

We have only ever supported a single instance of the pass-through authentication plugin (of any type) active in the server at once. In the 9.3 release, we are introducing a new aggregate pass-through authentication handler for use with the pluggable pass-through authentication plugin. This aggregate handler allows you to configure pass-through authentication so that it can support multiple external services, of the same or different types. You can now configure each of the pass-through authentication handlers to indicate which types of bind requests they support, and you can optionally attempt to use multiple handlers for the same bind operation under certain circumstances.

We have also added a new PingOne pass-through authentication handler that works in conjunction with the pluggable pass-through authentication plugin to allow passing through bind requests to PingOne. This handler offers essentially the same functionality as the PingOne pass-through authentication plugin, but the new PingOne handler allows it to be used in conjunction with the aggregate pass-through authentication handler so that pass-through authentication attempts can use PingOne in addition to other services.

More Options for Allowing Pre-Encoded Passwords

By default, the server does not allow clients to set pre-encoded passwords. This is something that we made it possible to disable via the allow-pre-encoded-passwords configuration property in the password policy, but we strongly discourage that because the server can’t perform any validation for pre-encoded passwords. A client could use this to set a password that doesn’t meet the server’s password strength requirements. And because most password storage schemes have many different ways of encoding the same password (which is important to protect against attacks using precomputed password dictionaries), this could also be exploited to allow a user to continue using the same password indefinitely, in spite of password expiration or other related settings.

The biggest risk to allowing pre-encoded passwords lies in self password changes rather than administrative password reset. Previously, administrators could override the server’s prohibition against pre-encoded passwords in one of two ways:

• If they are authorized to use the password update behavior request control, then they can use it to allow or reject pre-encoded passwords on a per-operation basis.

• If they have the bypass-pw-policy privilege, then they will be allowed to set pre-encoded passwords for other users (and do other things that the password policy configuration may prevent by default).

Previously, the allow-pre-encoded-passwords configuration property only offered two values: false (the default setting, in which the server would not allow clients to set pre-encoded passwords) or true (in which the server would allow any client to set pre-encoded passwords). In the 9.3 release, we are making it possible to use three additional values for this property:

• add-only — Indicates that the server will allow administrators to include pre-encoded passwords in add requests, but will continue to reject pre-encoded passwords for both self password changes and administrative password resets.

• admin-reset-only — Indicates that the server will allow administrators to perform an administrative password reset with a pre-encoded password, but will continue to reject pre-encoded passwords in add requests and for self password changes.

• add-and-admin-reset-only — Indicates that the server will allow administrators to include pre-encoded passwords in add requests and when performing administrative password resets, but will continue to reject pre-encoded passwords in self password changes.

In cases where an application may have a legitimate need to be able to set pre-encoded passwords for users, but it’s not feasible to update the application to use the password update behavior request control or to give the account is use the bypass-pw-reset privilege, these new options may make it possible for that application to set pre-encoded passwords for users while still preventing them for self password changes.

Directory REST API Support for More Password Operations

Updated the Directory REST API to add support for equivalents to the following LDAP extended operations:

• The standard password modify extended operation, which makes it possible to perform a self password change or an administrative password reset. Although you could previously change a user’s password by updating the password attribute, this operation offers a number of advantages, including:
  • You don’t need to know which attribute is used to store passwords in the target user’s entry.
  • There’s a dedicated field for supplying the user’s current password, which the password policy may require for self password changes.
  • You can optionally omit a new password and have the server automatically generate a new one and return it in the response.
  • It can be used in conjunction with a password reset token to allow a user to perform a self password change in cases where their account may be otherwise unusable.

• The proprietary get password quality requirements extended operation, which can be used to obtain a list of the requirements that new passwords will be required to satisfy, in both machine-parsable and human-readable forms.

• The proprietary generate password extended operation (which is called “suggest passwords” in the Directory REST API), which can be used to cause the server to generate suggested new passwords for a user.

New Password Validators

We have added a new disallowed characters password validator that makes it possible to reject passwords that contain any of a specified set of characters. You can define characters that are not allowed to appear anywhere in a password, as well as characters that are not allowed to appear at the beginning and/or the end of a password.

We have also added a new UTF-8 password validator that can be used to ensure that only passwords that are provided as valid UTF-8 strings will be allowed. You can optionally choose to limit passwords to only containing ASCII character or to allow non-ASCII characters, and you can also specify which classes of characters (e.g., letters, numbers, punctuation, symbols, spaces, etc.) should be allowed.

Support for ds-pwp-modifiable-state-json in Add Operations

If you enable the Modifiable Password Policy State Plugin in the server, then you can use the ds-pwp-modifiable-state-json operational attribute to set certain aspects of a user’s password policy state, including:

• Whether the account is administratively disabled
• Whether the account is locked as a result of too many failed authentication attempts
• Whether the account is in a “must change password” state
• The password changed time
• The account activation time
• The account expiration time

The password expiration warned time

Previously, the ds-pwp-modifiable-state-json attribute could only be used in a modify operation to alter the password policy state for an existing user. As of the 9.3 release, we now also allow it to be used in an add operation to specify state information for the account being created.

Fix Database Ordering With compact-common-parent-dn Values

The compact-common-parent-dn configuration property can be used to reduce the amount of disk space consumed by the database by tokenizing portions of entry DNs that lots of entries have in common. For example, in a database in which most of the entries are below “ou=People,dc=example,dc=com”, defining that as a compact-common-parent-dn value could reduce the size needed to store DNs of entries below that by up to 26 bytes. And this compaction can happen not only in the encoded representation of the entry, but also in an internal index that we use to map the DNs of entries to the identifier of the record that contains the data for that entry, so that can double the space savings.

Unfortunately, we discovered a bug in the way that the maintained that DN-to-ID database when custom compact-common-parent-dn values were specified. This bug could only appear under very specific circumstances, including:

• When one or more compact-common-parent-dn values were specified that are two or more levels below the base DN for the backend.

• When processing an unindexed search in which the base DN for the search is below the base DN for the backend, but above one or more of the compact-common-parent-dn values.

In such cases, the search could have incorrectly omitted entries from the search results that were below those compact-common-parent-dn values. This was due to an issue with the way that we ordered records in that DN-to-ID database.

We have fixed the problem in the 9.3 release. However, because the issue relates to the order in which records were stored in the database, if you are affected by this problem, then you will need to export the contents of the database to LDIF and re-import it. This isn’t something that you need to do unless you have configured compact-common-parent-dn values that are at least two levels below the base DN for the backend.

A Fix for Modifying ds-pwp-modifiable-state-json With Other Attributes

When we initially introduced support for the ds-pwp-modifiable-state-json operational attribute, we did not allow altering it in conjunction with any other attribute in the user’s entry. We also included an optimization in the plugin that handles that attribute so that if the requested change did not actually alter the user’s password policy state (e.g., because the new value only attempted to set state properties to values that they already had), the plugin would tell the server to skip much of the normal processing for that modify operation.

We have since updated the server to allow you to alter other attributes (except the password attribute) in the same request as one that modifies ds-pwp-modifiable-state-json. However, in doing so, we neglected to update the plugin so that it no longer skipped the remainder of the core modify operation processing if the ds-pwp-modifiable-state-json update did not result in any password policy state changes but there were still other, unrelated changes that should be applied. In such cases, the server could have failed to apply changes to those other attributes. This has been fixed, and other modifications in the request will still be processed even if a change to ds-pwp-modifiable-state-json does not alter the user’s password policy state.

Automatically Apply Configuration Changes to TLS Protocols and Cipher Suites

By default, the server automatically selects an appropriate set of TLS protocols and cipher suites that it will use for secure communication. This default configuration should provide a good level of security that avoids options with known weaknesses and that don’t support forward secrecy while still remaining compatible with virtually any client from the last fifteen years.

However, the server does allow you to explicitly configure the set of TLS protocols and/or cipher suites if you have a need to do so. Previously, making any such changes required you to either restart the affected connection handler or the entire server for them to actually take effect. As of the 9.3 release, these changes will now automatically take effect for the LDAP connection handler so that any new secure sessions that are negotiated after the change is made will use the updated settings.

Note that this is currently only supported for the LDAP connection handler. It is still necessary to perform a restart to make the change take effect for other types of connection handlers (like the HTTP or JMX handlers), or to make the change take effect for replication.

New Account Status Notification Types

We have added a new account-authenticated account status notification type that can be used to generate an account status notification any time a user authenticates with a bind operation that matches the criteria contained in an account status notification handler’s account-authentication-notification-result-criteria configuration property.

We have added a new account-deleted account status notification type that can be used to generate an account status notification any time a user’s account is removed with a delete operation that matches the criteria contained in an account status notification handler’s account-deletion-notification-request-criteria configuration property.

Fix a Replace Issue for Modifies of Attributes With Options

We have fixed an issue that could cause the server to behave incorrectly when processing a replace modification for an attribute that has some variants with attribute options in the target entry. If the replace modification does not include any values for the target attribute, the server would have previously removed all variants of the attribute from the entry, including those with and without attribute options. It will now correctly only remove the variant without any attribute options.

Improved Support for Passwords Containing Characters With Multiple Encodings

Unicode is an international standard that defines the set of characters that computers are intended to support. This includes a wide range of characters encompassing most written languages on earth, including not only the core set of ASCII characters used in English, but also characters with diacritical marks used in many Latin-based languages, non-Latin symbols like those used in Chinese hanzi and Japanese kanji, and even emojis.

In some cases, Unicode supports multiple ways of encoding the same character. For example, the “ñ” (Latin small letter N with tilde) character can be represented in two different ways:

• As the single Unicode character U+00F1
• As a lowercase ASCII letter n followed by Unicode character U+0303, which is a special combining mark that basically means “put a tilde over the previous character”

Previously, when encoding passwords, the Directory Server would always encode them using the exact set of UTF-8 bytes provided by the client in the request used to set that password, and the server would always use the exact set of UTF-8 bytes included in a bind request as a way of attempting to verify whether the provided password was correct. This approach works just fine in the vast majority of cases, but it has the potential to fail in circumstances where the password contains characters that have multiple Unicode representations, and the encoding used in the bind request is different from the encoding used when the password was originally set.

As of the 9.3 release, we have improved our support for authenticating with passwords that contain characters with multiple Unicode representations. If a password storage scheme indicates that a given plaintext password could not have been used to create the encoded stored password, but if the provided plaintext password contains one or more non-ASCII characters, then we will check to see if that password may have alternative encodings that could have been used to generate the stored password.

Improved Access Control Behavior for the Password Policy State Extended Operation

The password policy state extended operation provides the ability to retrieve and update a wide range of properties that are part of a user’s password policy state. In the 9.3 release, we have improved the way that it operates to avoid a common pitfall that has caused issues in the past for clients that are subject to access control restrictions.

Previously, whenever the extended operation handler retrieved the entry for the target user, it did so only under the authority of the authenticated user. If that user was subject to access control restrictions and didn’t have the permission to retrieve some or all of the operational attributes that are used to maintain password policy state information in a user’s entry, then the version of the entry that was retrieved would not contain those attributes, even if they were present. When using that entry to construct a view of the user’s password policy state, this might result in a state that is different from what the server would actually use when interacting with that account.

For example, if the target account has been locked as a result of too many failed authentication attempts, but the requester doesn’t have permission to see the attribute used to maintain information about those failed attempts, then the password policy state extended operation could report that the account was not locked even though it was. Note that this did not affect the server’s behavior when actually enforcing the account lockout, but it could still provide misleading or unexpected behavior for the client that issued the password policy state extended request.

As of the 9.3 release, we have changed the extended operation handler’s behavior to avoid this kind of problem. The server will still verify that the requester has permission to retrieve the target user’s account, but it will now re-retrieve that account using an internal root user that is not subject to password policy restrictions. This ensures that the extended operation handler will have access to all of the operational attributes that are needed to construct an accurate representation of the user’s password policy state.

Note that this new behavior really only affects attempts to retrieve information about a user’s password policy state. It does not have any effect on attempts to update that state. If the client attempts to use the password policy state extended operation to make a change to a user’s password policy state, but the requester does not have permission to write to the necessary operational attribute(s) in the target user’s entry, then the update attempt will continue to fail.

Configuration Properties for Managing the Configuration Archive

We have updated the config backend to add support for a few new configuration properties that can help better manage the configuration archive and reduce unnecessary bloat that it may cause. The new properties include:

• maintain-config-archive — Indicates whether the server should maintain a configuration archive at all. The configuration archive is maintained by default, and disabling it will not remove existing archived configurations.

• max-config-archive-size — Specifies the maximum number of archived configurations that the server should maintain. By default, this is unlimited. If this is enabled and there are more than the maximum allowed archived configurations, then the oldest configurations will be removed to make room for newer configurations.

• insignificant-config-archive-base-dn — Specifies base DNs for configuration changes that should not be preserved in the configuration archive. If a configuration change affects only affects entries below one of these base DNs, then it will not result in that change being maintained separately in the configuration archive. By default, we have included a value of “cn=topology,cn=config” so that changes to entries in the topology registry are excluded from the configuration archive. Certain updates to the topology registry, like adding a new replica into the topology, may result in a large number of changes that previously included a lot of bloat in the configuration archive.

Improved Substring Filter Handling for Logically Empty Substrings

Substring search filters are not allowed to contain empty (zero-length) substrings. If a client attempted to process a search with a substring filter that contains an empty substring, the server would have properly rejected it. However, there were cases in which the server did not properly handle substring search filters that contained non-empty substrings that normalized to empty strings (for example, a substring filter that targeted the telephoneNumber attribute in which one of the substrings contained only characters that are considered insignificant when matching telephone number values). The server would have incorrectly considered that normalized-to-empty substring as matching anything rather than nothing, and in some cases, that could cause the search to return unexpected matches. This has been corrected, and substring filters with substrings that normalize to empty values will now properly never match anything.

Improved Error Handling With Automatic Certificate-Based Authentication

By default, whenever a client presents their own certificate chain to the server during TLS negotiation and wants to use that certificate chain to authenticate, it needs to send a SASL EXTERNAL bind request to the server to cause it to perform the appropriate authentication processing. However, LDAP connection handlers offer an auto-authenticate-using-client-certificate configuration property that can cause them to attempt to automatically authenticate a client that presented its own certificate chain as soon as the TLS negotiation completes. Because this automatic authentication attempt happens without any explicit request from the client, there’s no way for the server to indicate whether it completed successfully.

Previously, if an automatic authentication attempt failed, the server would keep the connection alive, but in an unauthenticated state. This could yield unexpected behavior in applications that issued request with the expectation that they had authenticated, only to find those requests rejected due to access control restrictions. As of the 9.3 release, the server will now immediately terminate any client connection that presented its own certificate chain when auto-authenticate-using-client-certificate was set to true but the server was unable to successfully authenticate that client for some reason.

Improved Rebind Error Handling in the Directory Proxy Server

The Directory Proxy Server allows you to set an authorization-method value of rebind, which may be useful in cases where the backend server doesn’t support the intermediate client or proxied authorization request controls (e.g., Active Directory). In such cases, the Directory Proxy Server will remember the credentials that the client used to authenticate, and it will re-bind with those credentials before sending a request that should be authorized as that user.

Previously, if the rebind attempt failed for any reason, the Directory Proxy Server would have immediately reported a failure for the attempt to process a request that should have been authorized as that user. We have updated this behavior so that if the failure suggests that the underlying connection used for that attempt may no longer be valid, the server may re-try the attempt in a different server or on a newly created connection to the same server.

New Replication-Related Alert Types

We have defined a couple of new administrative alert types that the server can use to notify administrators of replication-related concerns:

• replication-missing-changes-risk — This alert type will be used if the server has developed a replication backlog that is large enough that the server is at risk of missing changes if it can’t catch up.

• replication-not-purging-obsolete-replicas — This alert type will be used when bringing replication online (most likely during server startup) if the replication-purge-obsolete-replicas configuration property is not set to true. This property is set to true by default as of the 9.2 release, so this primarily applies to older servers that have been updated. We strongly recommend enabling automatic purging of obsolete replicas to reduce unnecessary overhead in replication storage and network traffic, and this alert can be used to ensure that administrators are aware of this recommendation.

Encrypted PKCS #8 Private Keys

We have introduced support for encrypted PKCS #8 private key PEM files. This allows you to make use of private key files that don’t expose the key in the clear. Encrypted private keys require a password to access their contents.

It should be possible to use encrypted private key files anywhere that you can use an unencrypted private key file, including:

• When importing a certificate chain with manage-certificates import
• When exporting a private key with manage-certificates export-private-key
• When setting up the server with a certificate chain and private key obtained from PEM files
• When using replace-certificate to replace a listener or inter-server certificate with a certificate chain and private key obtained from PEM files

PKCS #11 Caching

The PKCS #11 key manager provider can be used to allow the server to obtain a listener certificate chain from a PKCS #11 token, like a hardware security module (HSM). Whenever the server needs to negotiate a new TLS session with a client, it can access the PKCS #11 token to identify the listener certificate chain that should be used for that processing. This processing may require multiple accesses to the PKCS #11 token. In cases where the PKCS #11 token is remotely accessed over a network, and especially when there is a notable network latency involved in that access, this can have a notable impact on the performance of the TLS negotiation process.

In the 9.3 release, we have introduced a new pkcs11-max-cache-duration property in the PKCS #11 key manager provider configuration. By setting this to a nonzero value, the server can use a degree of caching to eliminate the need for some of the interaction with the HSM, which can dramatically reduce the number of requests that need to be made to the PKCS #11 token.

Note that the use of caching does have a risk incorrect or unexpected behavior if the contents of the PKCS #11 token are altered so that the cached results are no longer accurate. As such, if you decide to enable caching, we recommend temporarily disabling that caching when making changes to the contents of the PKCS #11 token, and then re-enabling the caching once the changes are complete.

We have just released version 9.2.0.0 of the Ping Identity Directory Server. See the release notes for a complete overview of changes, but here’s my summary:

Potential Backward Compatibility Issues

• Removed support for incremental backups [more information]
• Updated the Groovy language version from 2.x to 3.x [more information]

Summary of New Features and Enhancements for All Products

• Added support for Java 17 [more information]
• Added support for accessing external services through an HTTP proxy server [more information]
• Added a Prometheus monitoring servlet extension [more information]
• Added support for authenticating to Amazon AWS using an IRSA role [more information]
• Added support for generating digital signatures with encryption settings definitions [more information]
• Updated replace-certificate when running in interactive mode so that it can re-prompt for a certificate file if the initial file existed but did not contain valid certificate data

Summary of New Features and Enhancements for the Directory Server

• Improved support for data security auditors [more information]
• Added new secure, connectioncriteria, and requestcriteria access control keywords [more information]
• Added support for defining resource limits for unauthenticated clients [more information]
• Added Argon2i, Argon2d, and Argon2id password storage schemes to supplement the existing Argon2 scheme [more information]
• Changed the default value of the replication-purge-obsolete-replicas global configuration property from false to true
• Updated migrate-ldap-schema to support migrating attribute type definitions from Active Directory in spite of their non-standards-compliant format
• Improved the usage text for the dsreplication enable command

Summary of New Features and Enhancements for the Directory Proxy Server

• Exposed the maximum-attributes-per-add-request and maximum-modifications-per-modify-request properties in the global configuration

Summary of New Features and Enhancements for the Synchronization Server

• Added support for synchronizing to SCIMv2 destinations [more information]
• Added a sync-pipe-view tool that can display information about the set of sync pipes configured in the server
• Added sync pipe monitor attributes related to account password policy state when synchronizing to a Ping Identity Directory Server

Summary of Bug Fixes

• Fixed an issue that could cause replication protocol messages to be dropped, potentially resulting in paused replication
• Fixed an issue in which a timeout could prevent adding servers to a large topology
• Fixed an issue in which an unexpected error could cause a replication server to stop accepting new connections
• Fixed an issue that prevented resource limits from being set properly for the topology administrator
• Fixed an issue in which the dsreplication tool incorrectly handled DNs in a case-sensitive manner
• Fixed an issue that could cause dsreplication enable to fail if there were any topology administrators without passwords
• Fixed an issue that could cause a configured idle timeout to interfere with replica initialization
• Fixed an issue that could prevent the server from generating an administrative alert when clearing an alarm that triggered an alert when it was originally raised
• Fixed an issue that could cause degraded performance to a PingOne sync destination
• Fixed an issue that could prevent users from changing their own passwords with the password modify extended operation if their account was in a “must change password” state and the request passed through the Directory Proxy Server
• Fixed an issue in which dsconfig would always attempt to use simple authentication when applying changes to servers in a group, regardless of the type of authentication used when launching dsconfig
• Fixed an issue that could cause certain kinds of Directory REST API requests to fail if they included the uniqueness request control
• Fixed an issue in which an unclean shutdown could cause the server to create exploded index databases
• Disabled the index cursor entry limit by default, which could cause certain types of indexed searches to be considered unindexed
• Fixed an issue that could adversely affect performance in servers with a large number of virtual static groups

Removed Support for Incremental Backups

We have removed support for incremental backups. This feature was deprecated in the 8.3.0.0 release after repeated issues that could interfere with the ability to properly restore those backups. These issues do not affect full backups, which continue to be supported.

As an alternative to full or incremental backups, we recommend using LDIF exports, which are more useful and more portable than backups. They are also typically very compressible and can be taken more frequently than backups without consuming as much disk space. Further, the extract-data-recovery-log-changes tool can be used in conjunction with either LDIF exports or backups to replay changes recorded in the data recovery log since the time the LDIF export or backup was created.

Updated the Groovy Language Version

In order to facilitate support for Java 17, we have updated the library providing support for the Groovy scripting language from version 2.x to 3.x. While this should largely preserve backward compatibility, there may be some issues that could prevent existing Groovy scripted extensions from continuing to work without any problems.

The only compatibility issue that we have noticed is that the 3.x version of the Groovy support library cannot parse Java import statements that are broken up across multiple lines, like:

import java.util.concurrent.atomic.
            AtomicLong;

This was properly handled in Groovy 2.x, but the Groovy 3.x library does not appear to support this. To address the problem, you will need to update the script to put the entire import statement on a single line, like:

import java.util.concurrent.atomic.AtomicLong;

If you have any Groovy scripted extensions, we strongly recommend verifying them in a test environment before attempting to update production servers.

Java 17 Support

We have updated the server to support running on JVMs running Java version 17, which is the latest LTS release of the Java language. Java versions 8 and 11 also continue to be supported.

Note that Java 17 support is limited to the Directory Server, Directory Proxy Server, and Synchronization Server. Java 17 is not supported for the Metrics Engine, although it continues to be supported on Java 8 and 11.

The best way to enable Java 17 support is to have the JAVA_HOME environment variable set to the path of the Java 17 installation when installing the server using either the setup or manage-profile setup commands. It’s more complicated to switch to Java 17 for an existing instance that was originally set up on Java 8 or 11 because there are changes in the set of JVM arguments that should be used with Java 17. As such, if you want to switch to Java 17, then we recommend installing new instances and migrating the data to them.

By default, installations using Java 17 will use the garbage first garbage collection algorithm (G1GC), which is the same default as Java 11. We also support using the Z garbage collector (ZGC) on Java 17, although we have observed that it tends to consume a significantly greater amount of memory than the garbage first algorithm. While ZGC can exhibit better garbage collection performance than G1GC, if you wish to use it, we recommend configuring a smaller JVM heap size and thoroughly testing the server under load and at scale before enabling it in production environments.

HTTP Forward Proxy Support

We have updated several server components to provide support for issuing outbound HTTP and HTTPS requests through a proxy server. Updated components include:

• The Amazon Key Manager cipher stream provider
• The Amazon Secrets Manager cipher stream provider, passphrase provider, and password storage scheme
• The Azure Key Vault cipher stream provider, passphrase provider, and password storage scheme
• The PingOne pass-through authentication plugin
• The PingOne sync source and destination
• The Pwned Passwords password validator
• The SCIMv1 sync destination
• The SCIMv2 sync destination
• The Twilio alert handler and OTP delivery mechanism
• The UNBOUNDID-YUBIKEY-OTP SASL mechanism handler

To enable HTTP forward proxy support for any of these components, first, create an HTTP proxy external server configuration object with a command like:

dsconfig create-external-server \
     --server-name "Example HTTP Proxy Server" \
     --type http-proxy \
     --set server-host-name:proxy.example.com \
     --set server-port:3128

You can also optionally use the basic-authentication-username and basic-authentication-passphrase-provider properties if the HTTP proxy server requires authentication.

Once the HTTP proxy external server has been created, update the target component to reference that server. For example:

dsconfig set-password-validator-prop \
     --validator-name "Pwned Passwords" \
     --set "http-proxy-external-server:Example HTTP Proxy Server"

Prometheus Monitoring Servlet Extension

We have added support for a new HTTP servlet extension that can be used to expose certain server metrics in a format that can be consumed by Prometheus or other monitoring systems that support the OpenMetrics data format. To enable it, add the servlet extension to the desired HTTP connection handlers and either restart the server or disable and re-enable those connection handlers. For example:

dsconfig set-connection-handler-prop \
     --handler-name "HTTPS Connection Handler" \
     --add "http-servlet-extension:Prometheus Monitoring" \
     --set enabled:false

dsconfig set-connection-handler-prop \
     --handler-name "HTTPS Connection Handler" \
     --set enabled:true

By default, the server is preconfigured to expose a variety of metrics. You can customize this to remove metrics that you don’t care about, or to add additional metrics that we didn’t include by default. Any single-valued numeric monitor attribute can be exposed as a metric. You can also customize the set of labels included in metric definitions, on both a server-wide and per-metric basis.

Improved AWS Authentication Support

The server offers a number of components that can interact with Amazon Web Services components,
including:

• A cipher stream provider that can use the Key Management Service
• A cipher stream provider, passphrase provider, and password storage scheme that can use the Secrets Manager

In the past, you could authenticate to AWS using either a secret access key or using an IAM role that is associated with the EC2 instance or EKS container in which the server is running. In the 9.2.0.0 release, we’re introducing support for authenticating with an IRSA (IAM role for service accounts) role. We are also adding support for a default credentials provider chain that can attempt to automatically identify an appropriate authentication method for cases in which the server is running in an AWS environment, or in cases where information about a secret access key is available through either environment variables or Java system properties.

To use the new authentication methods, first create an AWS external server that specifies the desired value for the authentication-method property. Then, reference that external server when creating the desired component. For example:

dsconfig create-external-server \
     --server-name AWS \
     --type amazon-aws \
     --set authentication-method:irsa-role \
     --set aws-region-name:us-east-2

dsconfig create-cipher-stream-provider \
     --provider-name KMS \
     --type amazon-key-management-service \
     --set enabled:true \
     --set aws-external-server:AWS \
     --set kms-encryption-key-arn:this-is-the-key-arn

Data Security Auditor Improvements

The server offers a data security auditor framework that can be used to iterate across entries in a number of backends and examine them for potential security-related issues or items of note. In the past, we’ve offered auditors that can do the following:

• Identify entries that define access control rules
• Identify accounts that have been administratively disabled
• Identify accounts that have passwords that are expired, are about to expire, or that have not been changed in longer than a given length of time
• Identify accounts that are locked as a result of too many authentication failures, because it’s been too long since the user last authenticated, or because they did not choose a new password in a timely manner after an administrative reset.
• Identify accounts with multiple passwords
• Identify accounts with privileges assigned by real or virtual attributes
• Identify accounts encoded with a variety of weak password storage schemes, including 3DES, AES, BASE64, BLOWFISH, CLEAR, MD5, RC4, or the default variant of the CRYPT scheme

In the 9.2 release, we’ve introduced support for several new types of data security auditors, including those that can do the following:

• Identify accounts with account usability errors, warnings, and/or notices
• Identify accounts that have an activation time in the future, an expiration time in the past, or an expiration time in the near future
• Identify accounts that have passwords encoded with a deprecated password storage scheme
• Identify accounts that have not authenticated in longer than a specified period of time, or that have not ever authenticated
• Identify accounts that reference a nonexistent password policy
• Identify entries that match a given search filter

We have also updated the Server SDK so that you can create your own data security auditors to use whatever logic you want.

In addition, we have updated the locked account data security auditor so that it can identify accounts that are locked as a result of attempting to authenticate with a password that fails password validator criteria, and we have updated the weakly encoded password data security auditor so that the following schemes are also considered weak: SMD5, SHA, SSHA, and the MD5 variant of the CRYPT scheme.

Finally, we’ve introduced support for a new audit data security recurring task that you can use to have the server automatically perform an audit on a regular basis.

New Access Control Keywords

We have introduced three new access control keywords.

The secure bind rule can be used to make access control decisions based on whether the client is using a secure connection (e.g., LDAPS or LDAP with StartTLS) to communicate with the server. Using a bind rule of secure="true" indicates that the ACI only applies to clients communicating with the server over a secure connection, while secure="false" indicates that the ACI only applies to clients communicating with the server over an insecure connection.

The connectioncriteria bind rule can be used to make access control decisions based on whether the client connection matches a specified set of connection criteria. The value of the bind rule can be either the name or the DN of the desired connection criteria.

The requestcriteria target can be used to make access control decisions based on whether the operation matches a specified set of request criteria. The value of the target can be either the name or the DN of the desired request criteria.

Note that because the Server SDK provides support for creating custom types of connection and request criteria, the introduction of these last two bind rules adds support for being able to define custom access control logic if the server’s existing access control framework doesn’t support what you want.

Resource Limits for Unauthenticated Clients

The server’s global configuration includes the following configuration properties that can be used to set default resource limits that will apply to all users that don’t have specific limits set for them:

• size-limit — Specifies the maximum number of entries that can be returned for a search operation
• time-limit — Specifies the maximum length of time the server should spend processing a search operation
• idle-time-limit — Specifies the maximum length of time that a client connection may remain established without any operations in progress
• lookthrough-limit — Specifies the maximum number of entries that the server can examine in the course of processing a search operation

These properties set global defaults for all clients, including those that aren’t authenticated. However, you may want to set lower limits for unauthenticated connections than for users that are authenticated. To make that easier to accomplish, we have added the following new additional properties that specifically apply to unauthenticated clients:

• unauthenticated-size-limit
• unauthenticated-time-limit
• unauthenticated-idle-time-limit
• unauthenticated-lookthrough-limit

By default, these properties don’t have any values, which will cause the server to inherit the value from the property that doesn’t specifically apply to unauthenticated clients (for example, if unauthenticated-size-limit is not set, then the server will use the size-limit value as the default for both authenticated and unauthenticated clients).

Improved Signature Generation

The server supports cryptographically signing log messages, backups, and LDIF exports. Previously, those signatures were always generated with MAC keys shared among other servers in the same topology. These keys are difficult to back up and restore, and the resulting signatures cannot be verified outside of the topology.
In the 9.2.0.0 release, we have updated the server so that it now generates digital signatures with encryption settings definitions. The server’s preferred definition will be used by default, but you can specify an alternative definition with the signing-encryption-settings-id property in the crypto manager configuration.

If digital signing is enabled but no encryption settings definitions are available, then a legacy topology key will continue to be used as a fallback.

Additional Argon2 Password Storage Schemes

The Argon2 key derivation function is a popular mechanism for encoding passwords, especially after it was selected as the winner of a password hashing competition in 2015. We introduced support for an ARGON2 password storage scheme in the 8.0.0.0 release.

There are actually three variants of the Argon2 algorithm:

• Argon2i — Provides better protection against side-channel attacks. The existing ARGON2 scheme uses this variant.
• Argon2d — Provides better protection against GPU-accelerated attacks.
• Argon2id — Mixes the strategies used in the Argon2i and Argon2d variants to provide a degree of protection against both types of attacks.

In the 9.2.0.0 release, we are introducing three new password storage schemes, ARGON2I, ARGON2D, and ARGON2ID, which provide explicit support for each of these variants.

Note that if you want to use the Argon2 algorithm to encode passwords, and you need to run in an environment that contains pre-9.2.0.0 servers, then you should use the existing ARGON2 scheme. The newer schemes should only be used in environments containing only servers running version 9.2.0.0 or later.

SCIMv2 Sync Destination

The Synchronization Server has included support for SCIMv1 servers as a sync destination since the 3.2.2.0 release. This support relies on an XML-based configuration to map LDAP source attributes to SCIM destination attributes.

In the 9.2.0.0 release, we’re introducing support for SCIMv2 servers as a sync destination. For this destination, all of the necessary configuration is held in the server’s configuration framework, so there is no need for a separate file with mapping information. This implementation introduces several new types of configurable components, including:

• HTTP authorization methods, which provide support for a variety of mechanisms for authenticating to HTTP-based services, including basic authentication and OAuth 2 bearer tokens (and in the latter case, you may configure either a static bearer token or have the server obtain one from an OAuth authorization server using the client_credentials grant type).
• A SCIM2 external server, which provides the SCIM service URL, authorization method, and other settings to use when interacting with the SCIMv2 service.
• SCIM2 attribute mappings, which describe how to generate SCIM attributes from the LDAP representation of a source entry.
• SCIM2 endpoint mappings, which associate a set of attribute mappings with an endpoint in the SCIMv2 server.
• The SCIM2 sync destination, which associates the SCIM2 external server and the SCIM2 endpoint mappings.

The documentation describes the process for configuring the Synchronization Server to synchronize changes to a SCIMv2 server. In addition, the config/sample-dsconfig-batch-files/configure-synchronization-to-scim2.dsconfig file provides an example that illustrates a set of changes that can be used to synchronize inetOrgPerson LDAP entries to urn:ietf:params:scim:schemas:core:2.0:UserM. SCIMv2 entries.