Add recursive LDAP parent group search (AD-style hierarchy across all LDAPs)

[EthanDieterich]

Overview

This pull request introduces improvements to Dex's LDAP connector to enable universal nested group search support across a variety of LDAP server implementations. It updates the code to allow recursive group membership discovery during user authentication and provides a more robust testing framework to validate the functionality.

This PR builds is largely based upon @paroque’s original #1058,

The groups() function in ldap.go was significantly refactored to add support for recursive nested group searching. Previously, the function directly built and executed LDAP search requests inline for each user matcher, then collected group names from the immediate search results. In the new version, the LDAP querying has been moved into a separate helper function called queryGroups(), which makes the main groups() function cleaner and easier to follow. A recursion flag and attribute have been added to the configuration. When recursion is enabled, after the initial group search, the function continues to search for parent groups using the specified recursion attribute. This process repeats until no new groups are found, allowing Dex to build a complete list of all direct and inherited group memberships. To prevent infinite loops and redundant results, the function checks for duplicate names and circular group references. If recursion is not enabled, the function simply returns the user’s direct group memberships, maintaining the original behavior.

To support per-matcher control over recursive searching, two new fields were added to the UserMatcher struct: Recursive and RecursionGroupAttr. The Recursive flag allows admins to specify whether recursion should be applied for a particular user matcher. The RecursionGroupAttr field lets them define which LDAP attribute should be used during the recursive lookup phase, allowing flexibility across different schemas. These assure that recursion is easily configurable and has broad compatibility.

To support the new recursive nested group functionality, a new unit test called TestNestedGroups was added. Using a new LDAP config and LDIF addition, this tests basic nested structures including a multi-level nesting and circular group references. It checks that users inherit all the correct group memberships through the recursive lookup process.

What this PR does / why we need it

Through extensive testing using both OpenLDAP and Apache Directory servers, we confirmed that Dex’s LDAP connector does not support nested group structures by default. While Dex could identify a user’s direct group memberships, it failed to recognize inherited group memberships through multi-level nesting, which are common in real-world LDAP environments. Although some systems like Microsoft Active Directory offer built-in nested group search (as cited in #1058 (comment)), many LDAP implementations do not, and we wanted to create a universal solution that would work across all implementations.

This PR introduces recursive group lookup functionality to Dex’s LDAP connector, allowing it to correctly discover and include all nested group memberships during authentication. Without this improvement for many LDAP implementations, users belonging to nested groups would not be authorized correctly, leading to access control issues for organizations relying on LDAP group hierarchies.

Our CI testing demonstrates the functionality with a series of example groups on the left:

Without the PR:

• John’s Groups: intermediateGroup
• Jane’s Groups: childGroup

With the PR and enabling recursion:

• John’s Groups: intermediateGroup, parentGroup
• Jane’s Groups: childGroup, intermediateGroup, parentGroup

The PR makes it so that after Dex finds a user’s direct groups, it can keep searching for any parent groups they’re part of until all group memberships, direct or indirect, are found.

On the right, we’re testing a circular reference scenario to ensure that the search correctly identifies both users as members of both circular groups without introducing duplicates or causing an infinite loop.

Finally, it is worth noting two more things:

1. We found that the AD-specific member:1.2.840.113556.1.4.1941: filter timed out after 2 minutes when searching for a
   Member’s groups in a real-world large corporate AD environment. Searching that same corporate AD for the same Member’s groups with Dex (including this PR’s functionality) takes 7 seconds, and correctly returns 300+ groups.

2. In addition to running the CI tests with OpenLDAP, we manually tested this new Dex functionality with Apache DS, where it also performs correctly.

Overall, these changes make Dex’s LDAP connector more flexible, more reliable across different environments, and better suited for real-world deployments that depend on complex group hierarchies.

[jsquyres]

Maintainers: we had a last minute-shuffle on this PR. It's now ready to review. Thanks!

[EthanDieterich]

Hello maintainers,
To accompany this feature PR, I've now also opened a documentation PR here: dexidp/website#196.

[jsquyres]

Ping; just checking in. Can someone review this PR? Thanks!

[jsquyres]

Periodic ping: we made this PR back on April 30. Can someone authorize the CI and give this a code review?

Thank you!

[nabokihms]

@EthanDieterich The PR requires a rebase now, otherwise CI checks will not run

[nabokihms]

Some minor configuration suggestions, otherwise looks good enough. I appreciate your patience.

[jsquyres]

Looks like the CI failure is indicating that this PR needs some kind of label. Is release-note/enhancement appropriate? (especially when paired with the corresponding docs update PR dexidp/website#196)

[nabokihms]

LGTM. Could you also update the doc PR as well? Thanks!

[as428y]

Hi
Tried this feature,
Having a lot of groups, the performance is not as good as expected.
For me, it took ~60 seconds to iterate over all groups I'm in

Maybe the following points will help

Replaced O(n²) duplicate detection with O(1) map lookup
Added circular reference protection
Multiple string comparisons
query batching

Possibly having multi-threading will improve it