Bug report: Web API endpoint search_gitlab_repos lists old component data instead of new

I found what I believe is a bug in one of the endpoint api/alm_integrations/search_gitlab_repos

It will return the GitLab project’s original values for

  • “sqProjectKey”: “x_y_z_int001_code_AXwmuUqhccjl3LgiYrr0”,
  • “sqProjectName”: “x.lips3.test.integrations.int001”

instead of the current ones (see e.g. components/show):

“component”: {
“key”: “x_y_z_ms001_code_AXwmuUqhccjl3LgiYrr0”

The two keys are two different components, but the api/alm_integrations/search_gitlab_repos is listing the old (incorrect one).

After some more resource, I found out that there were in fact two components that had the same GitLab project id and the older one was the one showing up in search_gitlab_repos instead of the current one.

I consider this a bug. The API should show the information for the current GitLab projects.

@ganncamp Just ping so that the thread does not close due to 7 days.

Hi @lfvjimisola ,

Thank you for reporting it. I’ll try to look closer at it sometime this week and will get back to you.

Best regards,
Bogdana

Any updates? I had a meeting with PO Python ecosystem who said that there was something going on behind the scenes (from what she could see), but nothing has been communicated here.

It’s a blocker for us since you, as expected, don’t get the accurate (i.e. up-to-date) information.

Hi @lfvjimisola,

When you say that:

Does this mean that the old one no longer exists? In the result, do you see just one component? Also, can you maybe share the full request/response? I can open personal messages for that.

Best regards,
Bogdana

Hi,

But, components exists in JFrog for the particular GitLab project id. But, in GitLab one no longer exists as the group id and artifact id in pom.xml has changed.

I’m not sure if I can reproduce since I had to remove the old component in order to be able to continue.

To reproduce this should work,

  1. create a minimal maven project, artifact id = artifact_org
  2. Create the project in SonarQube (GitLab linked)
  3. Run sonarscanner, confirm that there is data for the project in SonarQube
  4. Change the artifact id (artifact_new) in pom.xml
  5. Run sonarscanner

Now use the API (api/alm_integrations/search_gitlab_repos) and check which component the API returns. In our case it was artifact_org and not artifact_new.

BR,
Jimisola

1 Like

Could you please also clarify the following:

  1. What do you mean by artifact_org?
  2. Are you on Community Edition?
  3. How do you run sonarscanner?
  4. How do you use api/alm_integrations/search_gitlab_repos?

Best regards,
Bogdana

<groupId>com.mycompany.app</groupId>
<artifactId>artifact_org</artifactId>

is changed to e.g.

<groupId>com.mycompany.app</groupId>
<artifactId>artifact_new</artifactId>
  1. No, Developer Edition.

maven projects: maven plugin
python projects: pysonar-scanner (ping @jean.jimbo ;))
others: sonarscanner cli

In Java, like this but you can just use the endpoints directly to confirm.

private Mono<String> fetchProjectKey(int projectId, String projectPath, String projectName) {
		return webClient.get()
			.uri(uriBuilder -> uriBuilder.path("api/alm_integrations/search_gitlab_repos")
				.queryParam("almSetting", "LFV GitLab")
				.queryParam("p", "1")
				.queryParam("ps", 100)
				.queryParam("projectName", projectName)
				.build())
			.retrieve()
			.bodyToMono(Map.class)
			.doOnNext(response -> LOG.info("Response for search_gitlab_repos: {}", response))
			.flatMap(response ->

			{
				List<Map<String, Object>> repositories = (List<Map<String, Object>>) response.get("repositories");
				return repositories.stream().filter(repo -> repo.get("id").equals(projectId)).findFirst().map(repo -> {
					String fullSlug = repo.get("pathSlug") + "/" + repo.get("slug");
					LOG.warn("fullSlug: {}", fullSlug);
					if (!fullSlug.equals(projectPath)) {
						LOG.warn(
								"Project with ID {} has most likely changed name, token, key and/or URL as there is a difference in projectPath '{}' vs full slug '{}'",
								projectId, projectPath, fullSlug);
						return Mono.<String>empty();
					}
					return Mono.justOrEmpty((String) repo.get("sqProjectKey"));
				}).orElse(Mono.empty());
			});
	}

Hi @lfvjimisola,

I think I finally understand what might be the problem here, it’s sonar.projectKey, which isn’t explicitly specified. Your project key is a unique identifier for your project. If you are using Maven, ensure the key matches the “groupId:artifactId” format as it’s a default value for projectKey. When artifactId was changed, it created new SQ project, which isn’t bound to gitlab and won’t appear in api/alm_integrations/search_gitlab_repos results.
Please, let me know whether it makes sense.

Best regards,
Bogdana

As I explained to your colleague, we have created a system to auto-detect sonar key and token without the use of sonar-project.properties.

I could be wrong here, but I don’t think that you are right :thinking:

The new project is bound to GitLab because it shows if I delete the original component.
The bug here is that there are two components that have a link to the same GitLab project and, for the given endpoint, SonarQube returns the original (non-current) one.

Hi @lfvjimisola,

From the SQ side, we can’t be sure what the current project is for you. I’d suggest having one project under one Gitlab id and you can achieve it either by deleting the incorrect one or by unbinding it (Project Settings → DevOps Platform Integration).

Best regards,
Bogdana

I understand that you can’t be sure which is the current/correct project for the user, but your current “solution” is quite bad. Options are:

  1. as a user it is not obvious that GitLab ID are reused by SonarQube
  2. you are currently returning something whereas there are other BETTER options such as:
    a. return the id for the last seen, seen as in last one that you received data for (makes a lot more sense than for the oldest one)
    b. return all SonarQube projects linked to a GitLab project id (leave it to the user to decide which one is correct)
    c. raise an error message

BR,
Jimisola

Hi @lfvjimisola

Thank you for your ideas, we’ve created a ticket for this.

Best regards,
Bogdana

1 Like

I don’t have access to that Jira althought I have an account.

" xxx@lfv.se doesn’t have access to Jira on sonarsource.atlassian.net .

Can I get that somehow so that I can keep track of this ticket?

I think you’ll need to be logged out of the Atlassian cloud in order to view the ticket since you don’t belong to our Jira. I just opened the ticket in incognito mode so it’s definitely accessible to the public.

It’s not possible to track/watch the ticket without being a member of our Jira… and only we can be members of our Jira (licenses :confused:)

I can see it but I can’t track/watch which is what I wanted to do.

That said, this issue is really a blocker for us and has been for 6 months or so now.
To my knowledge there is no workaround either.

If you are able then it would be very useful to be able to information such as

  • created
  • last update

In order to determine which one that is the current one.

What is the likeliness of anything happening to this bug any time soon?

Hey there.

It might be an issue we can tackle before the upcoming LTA, but I can’t share any guarantee.

I understand the limitations on this one endpoint are frustrating for your automation, but the core functionality of SonarQube is still intact. This is something you’ll have to work around for now.

Ok. Fingers crossed it gets in before the upcoming LTA.

Well, define core and define intact…

And just to be clear, there is no workaround for this to my knowledge - I’ve spent hours trying out your APIs without any luck. If there is a workaround (e.g. internal API) then please advise.
But, I’ll go through the web interface again and see what API calls there is to sniff.

@Colin Hi, any updates on whether it can be tackled before the upcoming LTA?

Our SonarQube instance is exploding with redundant projects due to this bug and we have no workaround.

Hey @lfvjimisola

I don’t have any update to share on this ticket, and it doesn’t yet have a fix version.

Can you help me understand how the following options would prevent redundant projects from appearing on your instance?

Unfortunately, after all this time, I don’t really understand how the bug affects your automation.