Search API problem after upgrading 8.9 LTS -> 9.9 LTS

curl --user username:password -X GET -H "Content-Type: application/json" https://$SONARQUBE_URL/api/projects/search?ps=500&q=$PROJECT_NAME

Note that the issue is the same regardless of the page size parameter, whether it’s 500 or 5 (for example), and that successful hits of the API only return the one project being searched for (as intended).

When searching for all of our other projects, the api returns the expected result almost instantly. But when searching for these two specific projects, the api takes much longer than expected. In our troubleshooting and maintenance, I’ve observed that upon starting the server following database maintenance, the search API for these projects returns in a slow but usable timeframe of ~8-15 seconds, and that after several hours of the server being live, the response time slowed to 60+ seconds, at which time it hits a timeout limit (not sure what link in the chain this timeout is configured, could be on our load balancer, could be in sonarqube server, could be db query, etc.). It seems like this issue occurs regardless of user-generated load on the server, because I just tried the same queries on our test server which isn’t in use by our userbase and observed the same timeout problem.

Searching for other unaffected projects seems to remain quick, but these two projects become unsearchable via the API. I’ve noticed that I still have no problem searching for them in the web UI, though loading some tabs of the project page (e.g. Measures or Activity) are also quite slow.