Intermittent 500 errors from sonarcloud.io/api/alm_integration/list_repositories

SonarQube Cloud
1

Hey all,

We’re using this (admittedly undocumented) API endpoint to query for an existing project binding from our CI/CD pipelines: I can link to a post from a few months back that gives more context.

https://sonarcloud.io/api/alm_integration/list_repositories?organization=…

This endpoint is very slow to respond, and there’s no way to filter it to a specific repo: in fact, for GitLab it even includes archived repos, which for our organization are in the (high) hundreds, and this causes UI problems when adding a new project too. The return payload size is ~150k.

While this has been stable for a while, in recent weeks I’m seeing an increasing number of 500 errors, with body:

{
  "errors": [
    {
      "msg": "An unexpected error occurred. Please try again later."
    }
  ]
}

Typically I’ve dealt with this by just retrying the job, or incorporating an automated retry into the CI pipeline, but I’m now seeing the same errors recur over multiple retries.

Is there anything I can do to ameliorate this?

Cheers,

Jack.

2

As promised, a little more context is in this thread:

including guidance from folks at Sonar that this is the only API available today that fulfils this requirement, and hence the correct one to use.

The problem appears to be exactly this, acknowledged as a bug over three years ago!

Any updates on addressing this? I’m going to try the workaround suggested there, but whether it works for us will depend on if the error response for a project that’s already bound returns the project key.

3

Hi,

Using an undocumented endpoint means that it’s explicitly marked internal and thus there are no promises about functionality (or performance!).

Nonetheless, I’ll flag this for the team. But I wouldn’t hold my breath IIWY.

 
Ann

5

Thanks! As I linked to above, this API was recommended by Colin at Sonar; if there were a publicly-documented endpoint to do the same thing that’s absolutely what I would use.

For now I have followed the suggestion in the other thread I linked, to use:

https://sonarcloud.io/api/alm_integration/provision_projects

I’ve determined that if the project has already been imported, this returns the existing projectKey, which is exactly what I need.

One unfortunate side-effect of this is I may create a project just to delete it again. This is because if I determine a project should not be scanned, the pipeline removes it from Sonar; that has to be done by projectKey, so I need to do the previous provisioning step in order to get one of those from the installationKey (CI_PROJECT_ID, in the case of GitLab).

That niggle aside, the performance is massively improved from retrieving the full set of bound projects, and some of my later logic is simplified too, as provisioning takes care of project creation and SCM binding.

As you identified, the use of undocumented APIs definitely makes me edgy, and an alternative suggestion would be most welcome. The thread I linked to from a few months back goes into the required workflow in some detail, and at that time the recommendation from your side was the list_repositories API (which is where we came in).

Cheers,

Jack.

7

Hello @JackWhelpton ,

I am member of the development team.

I went through the thread(s) and I would like a clarification.

At this stage, what you are asking is whether there is an official API similar to the following, right?

GET https://api.sonarcloud.io/dop-translation/project-bindings?repositoryId=${CI_PROJECT_ID}

Cheers,
Chrislain

8

Hello,

I checked with my colleagues and there is no endpoint taking a CI/CD project id as a parameter.

Best,
Chrislain

10

Hey Chrislain,

Sorry for the belated reply… yep, just to confirm, what I was seeking was an API endpoint that would let me retrieve the Sonar projectKey (or an indication that the project is not bound) for an existing installationKey.

provision_projects does all of this for me, but with the unwanted side-effect of continuing to create the project if it doesn’t exist, which I want to make conditional on some other logic on my side.

1 Like