SonarQube not able to fetch DevOps repositories

[jfry2k (James R Fry)]

Must-share information (formatted with Markdown):

• which versions are you using (8.6.0.39681)

Greetings all,
We are in the process of setting up our SonarQube server for use. I am at the stage where I set up the PAT token to allow SonarQube to populate all of our projects in Devops in . The steps for this from the SonarQube Documentation is listed below for reference.

---

To import your Azure DevOps repositories into SonarQube, you need to first set your global SonarQube settings. Navigate to Administration > Configuration > General Settings > ALM Integrations , select the Azure DevOps tab, and click the Create configuration button. Specify the following settings:

** Configuration Name (Enterprise and Data Center Edition only) – The name used to identify your Azure DevOps configuration at the project level. Use something succinct and easily recognizable.*
** Azure DevOps server collection URL – Your full Azure DevOps Server collection URL. For example, https://ado.your-company.com/DefaultCollection .*
** Personal Access Token – An Azure DevOps Server user account is used to decorate Pull Requests. We recommend using a dedicated Azure DevOps Server account with Administrator permissions. You need a [personal access token] from this account with the scope authorized for Code > Read & Write for the repositories that will be analyzed. This personal access token is used for pull request decoration, and you’ll be asked for another personal access token for importing projects in the following section.*

---

However, some of the listed project repositories are showing an alert message which states "Could not fetch repositories for this project. Contact your system administrator, or [update your personal access token]. And some show the repository as expected.

Any ideas as to why it can’t get all the repositories for all projects?

[Belen_Pruvost (Belén Pruvost)]

Hi @jfry2k,

Thank you for raising this issue.

From what you are describing, this is the message you are encountering when trying to import an Azure DevOps Server project:

This could happen because of the following reasons:

1. The Azure Project does not have any repositories - which is very unlikely as it is not possible to delete all repositories on a Project from the UI
   
   
2. Because the PAT used does not have permission to read repositories for that given project.

Based on this, could you generate a new Azure token with Code > Read & Write permission?

Please make sure that the Azure DevOps User you are using to generate the token has access to the Project Repository you’re trying to see. User permissions on Azure can be configured on Collection Settings > Users > Manage user.

When this is done, delete the old Azure DevOps Server Integration Configuration on SonarQube ( Administration > Configuration > General Settings > ALM Integrations), and create one with the new token.

Finally, add the new personal access token for importing repositories following the steps listed on the documentation.

[jfry2k (James R Fry)]

Hi Belen,
Thanks so much for the reply. I have gone through the steps you listed and still get the same result.

I checked the permissions on my user account (Please note this is in Azure AD)

And I re-created the PAT with Full Access to all projects. (In Devops)

I went through the steps to create the ALM Integration and still get the same result.

Interestingly I did notice that when I go to check my user access level to all projects, some say “administrator” and some say “custom.” However, i can’t change “custom” to “administrator” and even if I could, the projects listed as custom aren’t all the same that it gives the fetch error for.

Any other ideas?

[Belen_Pruvost (Belén Pruvost)]

Hey @jfry2k,

Thank you for taking the time to follow these steps regarding the token generation.

Just to be sure, after you created the new PAT, did you go through these steps?

1. Deleting the old the ALM Integration configuration for Azure DevOps Server
2. Creating a new ALM Integration configuration:
   
   
3. Using the import feature for Azure DevOps Server
   
4. Providing the new PAT as part of the onboarding
   
   

If that’s the case, then let’s continue debugging together to understand where the issue might be.
I’m going to guide you through a list of steps, so that you can share with me the output and hopefully we can find together some hints.

1. Open SonarQube and go to the Import Azure DevOps Server page
   
2. Look for a Project name where the repositories are not being shown. In this example, it would be “Test Project”. Does the rendered name match the project name you see in Azure DevOps Server?
   
3. If that’s the case, then let’s take a look at what the API is returning. On your browser, open the Network tab. If you’re using Google Chrome, you can access it by pressing F12. Clear the previous history of network calls, and type in the word “search” to filter for the right SQ endpoint.
   
   
4. Expand the accordion of the Azure DevOps Project, so that repositories (or lack of them!) can be listed. You should see a new record being listed on the Networks tab of your browser.
   
   
5. Could you confirm if on the Status column of the Network record you are seeing 200?
6. Finally, click on the name of the Network record and then on “Response”. What is being returned here?
   
   

Thanks in advance for following this.

[srinivas7y (Srinivasa Rao Y)]

Hi Belen,

Even I’m facing same issue. I followed all the steps provided by you. Still there is no luck. For all the projects I’m getting same error. Please find the attached screenshots and do needful.

Using Sonarqube community edition. Version: 8.9.1.44547

[Belen_Pruvost (Belén Pruvost)]

Hi @srinivas7y ,

First, let’s try to see if your ADO token can actually list repositories for a given project. For that, you need to base64 encrypt the token.

To do this, you can use this page. The text you need to encode is accessToken:your_token

For example, if you token was 123, you would need to encode accessToken:123, which would result in YWNjZXNzVG9rZW46MTIz

Then, let’s try to make a curl request to your Azure instance to list the repositories:
curl -H "Authorization: BASIC <encrypted value>" "<your ADO instance url including collection name>/<your project>/_apis/git/repositories?api-version=3.0"

If my instance url is https://ado.com, my collection is DefaultCollection and my project is Test_Project, the curl request is going to be
curl -H "Authorization: BASIC YWNjZXNzVG9rZW46MTIz" "https://ado.com/DefaultCollection/Test_Project/_apis/git/repositories?api-version=3.0"

Does this request return the repositories you would be expecting to see in SQ?

[srinivas7y (Srinivasa Rao Y)]

Thanks for your response Belen. No still not able to list the repositories using above method also. I tried with 2 Azure admin accounts. Please find the attached screenshot. If you’re available in teams or zoom we can have a quick call.

[Belen_Pruvost (Belén Pruvost)]

In this case, Azure API is no showing any repos for project TracX 2.0 or TracX. Either there are no repositories, or the token is invalid for listing them.

In Azure, can you browse within those projects with the user you’re generating the tokens with, and see any repositories?

[srinivas7y (Srinivasa Rao Y)]

Yes Belen. I’m able to see branches, files and folders inside those repositories. These projects are there from more than 5+ years. Accounts which I’m trying for sonarqube integration r admin accounts. Even we implemented CI CD pipeline with these accounts. Recently we thought of integrating sonarqube that is where we r getting this issue, Once our initial testing is done we are planning to opt for developer or enterprise edition.

[Belen_Pruvost (Belén Pruvost)]

@srinivas7y thanks for sharing this.

Are you using Azure DevOps Services? Or Azure DevOps Server? If it’s the latest, which version are you using?

Could you create a new token with full access, and try the curl request again? If nothing is returned still, could you attempt the same request but without the api-version flag?

[srinivas7y (Srinivasa Rao Y)]

We are using Azure DevOps Services only.

Created new token with full access & tried with both scenarios still issue is same.

[srinivas7y (Srinivasa Rao Y)]

Hi Belen,

Just now I tried with my personal account also. I’m getting same error. Unable to find what will be root cause. In my personal account I have only one user account and which has admin privileges. This is also an Azure DevOps Service

[Belen_Pruvost (Belén Pruvost)]

Hey @srinivas7y ,

One thing you would need to make sure is that the generated Personal Access Token is created for the workspace for which you are trying to list repositories.

SQ relies on Azure’s API to list repos, you can find more details about the List Repositories endpoint here.

In this case, there could be some misconfiguration on Azure side. I suggest you ask Azure for support, and once the root cause is explained, we can take a look together at what’s happening on SQ side.

[srinivas7y (Srinivasa Rao Y)]

@Belen_Pruvost , As of now Azure DevOps support is not available.

1. Personal Access Token can be created based on user level based on my understanding. If I login with same user I’m able to see all the projects, repositories, branches, files and folders.
2. I tried with my personal Azure DevOps account as well with my personal project, even there I’m getting same issue.
3. In my personal Azure DevOps account I have only user account (admin) which has full control over the repository I have. Here I have only one Project and inside I have 3 repositories.

Any suggestions would be really helpful. As I mentioned in my previous message if you’re available for a quick call then we can have a call in Teams or Zoom.

[Belen_Pruvost (Belén Pruvost)]

About point 1, when you’re creating the Personal Access Token in https://dev.azure.com/test-organization/_usersSettings/tokens, you have the option to select to which organization it will have access to:

[srinivas7y (Srinivasa Rao Y)]

Yes correct. Here in both Company and my personal Azure DevOps accounts we have only one Organization available. But I have created 2 tokens with full access. 1 token by selecting the organization which we have and 1 token I created by selecting All accessible organizations. But still our issue is same.

[srinivas7y (Srinivasa Rao Y)]

@Belen_Pruvost , I contacted Azure DevOps support team.

Issue is Sonarqube is listing repositories which are based on GIT. It is not listing repositories which are created by using TFVC or Visual Studio.

So, now my requirement are

1. TFVC + Sonarqube Integration should work. Can you tell me is there any plugin available to list repositories which are created using TFVC.
2. Sonarqube should run as a windows service instead of starting it manually using StartSonar.bat batch file. So that even is server restart happens it will start automatically.

[Belen_Pruvost (Belén Pruvost)]

Hey @srinivas7y, SonarQube does not support TVFC. You can find more details about supported SCMs here.

Kind regards,
Belén

[srinivas7y (Srinivasa Rao Y)]

Thanks for your response @Belen_Pruvost . I’m looking into other alternatives. If you find any plugin pls do let me know. Can you guide me on 2nd point ? Or If you have any script available for this kindly share respective link.