Improve the performance when using enumeration #8395
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request effectively improves performance by optimizing logging, removing an unused function, and correcting a condition related to enumeration. The changes are well-aligned with the stated goals of enhancing enumeration performance, especially for large user bases. The addition of a new test case for the general enumeration scenario ensures that the modified logic is adequately covered.
| return ret; | ||
| } | ||
|
|
||
| int sysdb_enumpwent(TALLOC_CTX *mem_ctx, |
There was a problem hiding this comment.
Also
db/sysdb.h:875:int sysdb_enumpwent(TALLOC_CTX *mem_ctx,
?
There was a problem hiding this comment.
Just removing unused code. But I have no hard feelings about it. I can remove that commit if it is better to keep this function.
There was a problem hiding this comment.
I mean, this commit should also remove function declaration from the header.
src/db/sysdb_search.c
Outdated
| DEBUG(SSSDBG_TRACE_LIBS, "Searching timestamp entries with [%s]\n", | ||
| dn_filter); | ||
|
|
||
| DEBUG(SSSDBG_TRACE_LIBS, "Searching timestamp entries (filter can be long)\n"); |
There was a problem hiding this comment.
I suggest keeping filter but with truncation using %.ms format syntax.
There was a problem hiding this comment.
Done. Limited to 50 characters.
|
Mistype in the commit message: "We must look into de TS cache" |
Function sysdb_enumpwent() is not used. It was replaced by sysdb_enumpwent_filter().
When there are too many users (17,000+) this message can be too long. Limit it to the first 50 characters. Resolves: SSSD#6951
We must look into the TS cache only when a name is provided. Using the TS cache on an unfiltered enumeration is useless. Resolves: SSSD#6951
Added a case that was not checked before. It is the case when `attr`, `attr_name` and `addtl_filter` are all `NULL`.
Fixed. |
3 participants
This PR includes:
Enumeration, specially when there are 15,000+ users, is slow. This fix helps, but it doesn't work miracles.
In my test environment, the enumeration went from 8 minutes to about 1.
It is important to know that, with such an amount of users, many operations time out. It is necessary to increment the
timeoutin[nss]and for the domain, but also set large values forldap_enumeration_refresh_timeoutandldap_search_timeoutin the domain. I used these values to avoid any timeout (YMMV):