How to find the duplicated code using SonarQube API

Template for a good bug report, formatted with Markdown:

  • versions used (SonarQube Enterprise Edition
    Version 8.2 (build 32929))
  • There are api available to find number of code duplications , number of files have duplication and number of duplicate lines using metrices but there is no way to find what;s that duplicate code and name of files with line number
  • steps to reproduce - https://url of sonarqube/api/measures/component?component=projectkey-20200319&metricKeys=duplicated_files
  • potential workaround - Not Known

#Api #codeduplicate

P.S.: use the #bug:fault sub-category if you’re hitting a specific crash/error , or the #bug:fp sub-category for rules-related behaviour

Hi @naveenaggarwal1,
It is a bit tricky to get the invdividual figures from SonarQube.
The overall duplication rate is comparatively easy, when you query the following URL (it should show number of bugs and duplicated lines ratio:
https://<sonar-server-url>/api/measures/component?component=<your-id>&metricKeys=bugs%2Cduplicated_lines_density

For the other measures your should:

  1. Logon to SonarQube with your browser
  2. Open the developer tools (hit <F12> on your keyboard)
  3. Browse to the page in SonarQube and monitor the request in the Developer Tools.

In the picture below, you can see a request, that reveals individual files and their duplication ratio

The issues/search API gives you the capability to search for issues against specific rules. You could search based on the duplicate code rules. The results include a lot of information about the issue including component and line. That might get you right to the duplication.

Thanks @bugbouncer for your response. I am more looking from automation perspective and don’t want to login SonarQube UI. I am looking for API support which give all details for code duplication including number of lines, names of files and line numbers which has code duplication with duplicate code block as well. I intend to generate an automated report with this all details.

Thanks @RSchaeferHIG for your response. I will try this and see if i can get code duplication file name with duplicated code blocks using search API.

Hi @RSchaeferHIG i can’t find the rule for duplicate code rule for issue search API ? Do you mind sharing the API call for issue/search API and all parameters for finding duplicate code files with details. Thanks in advance,

I was mistaken. I thought duplications had a rule associated with them. They do not. I’ll poke at the API to see if I can find a way to get duplication info.

Thanks @RSchaeferHIG

Hello @naveenaggarwal1,
I got your point.
My hint was actually: Look how the SonarQube-UI uses the REST-API to retrieve information from the backend.
The SonarQube-UI uses the same REST-API as you intend to use.
So if you see a data item in the UI, you can use my method to find the respective REST call and use it for your automation purposes.

Hi @bugbouncer i understand the solution you are suggesting. But i am finding below issues in that solution.

In SonarQube UI i can view the code duplicate metrics as below

image


For finding the details of duplicate lines i have to click on duplicate lines which will browse the project structure on right hand side for seeing the actual file name with duplicate code i have to click on each and every folder and for each API call for click the URl becomes so complex. replicating same scenario in API automation is very complex. What I was thinking as this information is already there in SonarQube database there should be some direct way of retrieving same using API. I hope you understand how complex it could be.

There are APi calls which give you metrices for code duplication as below code . I am looking similar API for getting details of duplication which has file name and line numbers of duplicate files.

$metricname = “duplicated_files”
$url = $SonarQubeQualityGateURL + “/measures/component?component=$projectkey&metricKeys=” + $metricname
Write-Host $url
$response= Invoke-RestMethod -Uri $url -Headers $headers
Write-Host “Number of Duplicate files=” $response.component.measures | Format-Table

$metricname = “duplicated_lines_density”
$url = $SonarQubeQualityGateURL + “/measures/component?component=$projectkey&metricKeys=” + $metricname
Write-Host $url
$response= Invoke-RestMethod -Uri $url -Headers $headers
Write-Host “Duplicate files Density=” $response.component.measures -ForegroundColor Red

$metricname = “duplicated_lines”
$url = $SonarQubeQualityGateURL + “/measures/component?component=$projectkey&metricKeys=” + $metricname
Write-Host $url
$response= Invoke-RestMethod -Uri $url -Headers $headers
Write-Host “Number of duplicate lines=” $response.component.measures -ForegroundColor Red

You are probably looking for GET api/duplications/show

Thanks @Colin_SonarSource yes dats correct I am looking for same api (https://codeen-app.euclid-ec.org/sonar/web_api/api/duplications) but I am not getting any output when i am hitting this api but there are duplication in my project which I am able to see using other metrices api. Am i missing anything ?

https://mycompanyinstance/api/duplications/show?key=Projectkey

Output is always blank as below

Number of duplicate blocks= @{duplications=System.Object[]; files=}

duplication object and number of files is always blank.

You’ll need to provide a specific file.

For example:

// 20200526220438
// http://localhost:9000/api/duplications/show?key=org.sonarsource.php%3Aphp%3Aphp-frontend%2Fsrc%2Fmain%2Fjava%2Forg%2Fsonar%2Fphp%2Ftree%2Fimpl%2Fexpression%2FLiteralTreeImpl.java

{
  "duplications": [
    {
      "blocks": [
        {
          "from": 39,
          "size": 25,
          "_ref": "1"
        },
        {
          "from": 38,
          "size": 25,
          "_ref": "2"
        }
      ]
    }
  ],
  "files": {
    "1": {
      "key": "org.sonarsource.php:php:php-frontend/src/main/java/org/sonar/php/tree/impl/expression/LiteralTreeImpl.java",
      "name": "php-frontend/src/main/java/org/sonar/php/tree/impl/expression/LiteralTreeImpl.java",
      "uuid": "AXJSV6bHmYM8o0DG8TZc",
      "project": "org.sonarsource.php:php",
      "projectUuid": "AXJSV5XtWfZtOXuuH8nd",
      "projectName": "SonarPHP"
    },
    "2": {
      "key": "org.sonarsource.php:php:php-frontend/src/main/java/org/sonar/php/tree/impl/expression/ExpandableStringCharactersTreeImpl.java",
      "name": "php-frontend/src/main/java/org/sonar/php/tree/impl/expression/ExpandableStringCharactersTreeImpl.java",
      "uuid": "AXJSV6bHmYM8o0DG8TZI",
      "project": "org.sonarsource.php:php",
      "projectUuid": "AXJSV5XtWfZtOXuuH8nd",
      "projectName": "SonarPHP"
    }
  }
}

It might also be useful to know the Internal Web API GET api/sources/lines provides duplication data per line as well.

Thanks @Colin_SonarSource my scenario is different I need to find all files for a given project which has duplicate code . I want to create a report using automation in Powershell. I don’t have name of files to pass as parameter to this api. I want to get all information using api automation without any manual step.