Slow project creation via web/api

SonarQube Server / Community Build
1
  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension): 9.7.1
  • how is SonarQube deployed: zip

Hi. After upgrading from 8.9.7LTS to 9.7.1 faced a problem: creating of projects takes 5-10min (before upgrade its take 5-10sec).
After some research found then all time spends for SELECT request in components table (37mill rows)

2023-05-10 16:58:39.218 > LOG:  duration: 138485.207 ms  execute <unnamed>: SELECT
 
            p.uuid as uuid,
            p.uuid_path as uuidPath,
            p.branch_uuid as branchUuid,
            p.module_uuid as moduleUuid,
            p.module_uuid_path as moduleUuidPath,
            p.main_branch_project_uuid as mainBranchProjectUuid,
            p.kee as kee,
            p.name as name,
            p.long_name as longName,
            p.description as description,
            p.qualifier as qualifier,
            p.scope as scope,
            p.language as language,
            p.root_uuid as rootUuid,
            p.path as path,
            p.enabled as enabled,
            p.copy_component_uuid as copyComponentUuid,
            p.private as isPrivate,
            p.created_at as createdAt
 
            FROM components p
            where
              lower(p.kee)=lower($1)
< 2023-05-10 16:58:39.218 > DETAIL:  parameters: $1 = 'test_mdo'

I think its kind of bug

2

Hi,

Thanks for this report. Unfortunately, 9.7.1 is already EOL. Could you upgrade to a supported version: 9.9.1 LTS or 10.0, and see if this is replicable?

 
Ann

3

Hi.
We are using SonarQube 10.0. There are more than 20,000 projects.
It takes 8-18 sec to create a project via api.
POST api/projects/create?name=projectName&project=project

It seems that it takes less time when project is created manually, via web：
https://sonarqube.***/projects/create?mode=manual

Is there any way to reduce the creating time via api？

1 Like
4

Hi @liuxl,

I know your question seems related, but since a lot happened between 9.7 and 10.0, can you create a new thread, please? Include your DB flavor and any relevant-seeming server logs.

 
Ann

5

Thanks for your reply.

We also found that the SELECT (listed by Dmitriy above) takes too much time. The components table does not have correct index on the search key.

After creating the index, it only takes 2ms to complete the same SELECT.

Perhaps it is related to the DB version? The index may be missed from the upgrade?

7

Hi @liuxl

Could you please let us know what DB version you use, and what index you created?

9

DB Version Postgres.10.3
We upgraded sonarqube from version 9.5 to 9.9, and to 10.0.

In DB, we found some indexs on Table components ,but no index about the lower(p.kee).

11

Sure. Already upgraded to 9.9.1 LTS and Postgresql 13.9
Same project creation time

14

Hello @Deadstar2011 thanks a lot for the extra details, we were able to reproduce your issue and track it down to the use of the lower function in the query you posted.

We created a Jira for tracking this.

Regards,
Matteo

2 Likes