How can I do to SonarQube analyzes the PR and show just the New Code in decoration

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
    SonarQube 9.6 - Developer Edition

  • what are you trying to achieve
    I’m running a script that clones my repository from Github and sends a trigger to SonarQube to analyze my java project. If SonarQube analyzes a branch everything works well, but when I try to analyze a PR, SonarQube runs the task but sends to Github the Overall Code decoration.

  • what have you tried so far to achieve this
    I’ve tried to set the SCM provider, changed the New Code configuration.

  • observation:
    I have a warning about SCM provider autodetection failure. But when I try to set the parameter “sonar.scm.provider” I receive the error “Not inside a Git work tree” and even when I put the “sonar.projectBaseDir” the error keeps it.

Hey there.

Perhaps you can share the script / logs from the output of the script in the context of a pull request analysis.

@Colin,

This is my script.

GIT_USER=my-git-user
GIT_TOKEN=my-git-token

rm -rf xxx
if [ ! -d xxx ]; then
    git clone https://$GIT_USER:$GIT_TOKEN@github.com/xxx.git
fi
rm -rf zzz
if [ ! -d zzz ]; then
    git clone https://$GIT_USER:$GIT_TOKEN@github.com/zzz.git
fi

GITHUB_REPO=xxx
SONAR_HOST=my-sonarqube-address
SONAR_LOGIN=my-sonarqube-login
SONAR_PASSWORD=my-sonarqube-password
PR=pr-number
PR_BRANCH=pr-branch-name
BASE_BRANCH=main-branch

if [ "$GITHUB_REPO" = "xxx" ]; then
    cd xxx
fi

if [ "$GITHUB_REPO" = "zzz" ]; then
    cd zzz
fi

git status -b

JAVA_HOME=/opt/jdk-17
export MAVEN_OPTS="-Xms512m -Xmx4g"

# run the analysis
mvn clean verify sonar:sonar -Dmaven.test.failure.ignore=true \
-Dsonar.java.jdkHome=/opt/jdk-17 \
-DskipUTs=true \
-Dsonar.host.url=$SONAR_HOST \
-Dsonar.login=$SONAR_LOGIN \
-Dsonar.password=$SONAR_PASSWORD \
-Dsonar.verbose=true \
-Dsonar.pullrequest.key=$PR \
-Dsonar.pullrequest.branch=$PR_BRANCH \
-Dsonar.pullrequest.base=$BASE_BRANCH
#-Dsonar.scm.provider=git \
#-Dsonar.log.level=trace

And this is the log:
|build|28-Sep-2022 15:32:02|[INFO] SonarQube version: 9.6.1.59531|
|build|28-Sep-2022 15:32:02|[INFO] Default locale: en, source code encoding: UTF-8|
|build|28-Sep-2022 15:32:03|[INFO] Load global settings|
|build|28-Sep-2022 15:32:03|[INFO] Load global settings (done) | time=217ms|
|build|28-Sep-2022 15:32:03|[INFO] Server id: xxxxxxxxxxxxxx|
|build|28-Sep-2022 15:32:03|[INFO] Load/download plugins|
|build|28-Sep-2022 15:32:03|[INFO] Load plugins index|
|build|28-Sep-2022 15:32:03|[INFO] Load plugins index (done) | time=55ms|
|build|28-Sep-2022 15:32:06|[INFO] Load/download plugins (done) | time=2542ms|
|build|28-Sep-2022 15:32:06|[INFO] Loaded core extensions: developer-scanner|
|build|28-Sep-2022 15:32:06|[INFO] Process project properties|
|build|28-Sep-2022 15:32:06|[INFO] Process project properties (done) | time=83ms|
|build|28-Sep-2022 15:32:06|[INFO] Execute project builders|
|build|28-Sep-2022 15:32:06|[INFO] Execute project builders (done) | time=1ms|
|build|28-Sep-2022 15:32:06|[INFO] Project key: xxxxxxxx|
|build|28-Sep-2022 15:32:06|[INFO] Base dir: /home/my-project|
|build|28-Sep-2022 15:32:06|[INFO] Working dir: /home/my-project/project/target/sonar|
|build|28-Sep-2022 15:32:06|[INFO] Load project settings for component key: ‘xxxxx’|
|build|28-Sep-2022 15:32:06|[INFO] Load project settings for component key: ‘xxxxx’ (done) | time=28ms|
|build|28-Sep-2022 15:32:06|[INFO] Load project branches|
|build|28-Sep-2022 15:32:06|[INFO] Load project branches (done) | time=35ms|
|build|28-Sep-2022 15:32:06|[INFO] Load project pull requests|
|build|28-Sep-2022 15:32:06|[INFO] Load project pull requests (done) | time=28ms|
|build|28-Sep-2022 15:32:06|[INFO] Load branch configuration|
|build|28-Sep-2022 15:32:06|[INFO] Found manual configuration of branch/PR analysis. Skipping automatic configuration.|
|build|28-Sep-2022 15:32:06|[INFO] Load branch configuration (done) | time=1ms|
|build|28-Sep-2022 15:32:06|[WARNING] SCM provider autodetection failed. Please use sonar.scm.provider to define SCM of your project, or disable the SCM Sensor in the project settings.|
|build|28-Sep-2022 15:32:07|[INFO] Load quality profiles|
|build|28-Sep-2022 15:32:07|[INFO] Load quality profiles (done) | time=50ms|
|build|28-Sep-2022 15:32:07|[INFO] Load active rules|
|build|28-Sep-2022 15:32:08|[INFO] Load active rules (done) | time=1730ms|
|build|28-Sep-2022 15:32:08|[INFO] Load analysis cache|
|build|28-Sep-2022 15:32:10|[INFO] Load analysis cache | time=1632ms|
|build|28-Sep-2022 15:32:10|[INFO] Pull request 8556 for merge into MAIN from sonarqube-test-pr|
|build|28-Sep-2022 15:32:10|[INFO] Load project repositories|
|build|28-Sep-2022 15:32:10|[INFO] Load project repositories (done) | time=190ms|
|build|28-Sep-2022 15:32:10|[INFO] Indexing files…|

|build|28-Sep-2022 15:47:32|[INFO] CPD Executor CPD calculation finished (done) | time=2866ms|
|—|—|—|
|build|28-Sep-2022 15:47:33|[INFO] Analysis report generated in 858ms, dir size=112.4 MB|
|build|28-Sep-2022 15:47:46|[INFO] Analysis report compressed in 13138ms, zip size=45.7 MB|
|build|28-Sep-2022 15:47:46|[INFO] Analysis report generated in /home/my-project/project/target/sonar/scanner-report|
|build|28-Sep-2022 15:47:48|[INFO] Analysis report uploaded in 1959ms|
|build|28-Sep-2022 15:47:48|[INFO] ANALYSIS SUCCESSFUL, you can find the results at: https://redacted.com/dashboard?id=my-project%3Aproject&pullRequest=8556|
|build|28-Sep-2022 15:47:48|[INFO] Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report|
|build|28-Sep-2022 15:47:48|[INFO] More about the report processing at https://redacted.com/api/ce/task?id=zzzzzzzzz|
|build|28-Sep-2022 15:47:49|[INFO] Analysis total time: 15:42.660 s|

And what was the response of git status -b when it ran directly before this?

The response is:

build 28-Sep-2022 15:26:33 On branch sonar/8556
build 28-Sep-2022 15:26:33 nothing to commit, working tree clean

@Colin,

Do you have some idea why this is going on?

Thanks,

Hello @flima33,

In your script you first have an assertion based on the arguments if [ ! -d zzz ]; then, and then there is this hardcoded assignment: GITHUB_REPO=xxx

So it looks like whatever is passed as argument is then overridden by the latter, which could result in unexpected behaviour (you clone project zzz but then the script tries to cd to xxx).

To exclude any problem with the logic, could you please try to execute a simplified version of your script:

GIT_USER=my-git-user
GIT_TOKEN=my-git-token

rm -rf xxx

git clone https://$GIT_USER:$GIT_TOKEN@github.com/xxx.git

SONAR_HOST=my-sonarqube-address
SONAR_LOGIN=my-sonarqube-login
SONAR_PASSWORD=my-sonarqube-password
PR=pr-number
PR_BRANCH=pr-branch-name
BASE_BRANCH=main-branch

cd xxx

pwd
git status -b

JAVA_HOME=/opt/jdk-17
export MAVEN_OPTS="-Xms512m -Xmx4g"

# run the analysis
mvn clean verify sonar:sonar -Dmaven.test.failure.ignore=true \
-Dsonar.java.jdkHome=/opt/jdk-17 \
-DskipUTs=true \
-Dsonar.host.url=$SONAR_HOST \
-Dsonar.login=$SONAR_LOGIN \
-Dsonar.password=$SONAR_PASSWORD \
-Dsonar.verbose=true \
-Dsonar.pullrequest.key=$PR \
-Dsonar.pullrequest.branch=$PR_BRANCH \
-Dsonar.pullrequest.base=$BASE_BRANCH

and if it still does not work, can you send us the full output of this simplified script, together with which git implementation/version you are using?

also, can you make sure that the .git folder exists in the cloned project?

Thanks,
Aurélien

Hi @aurelien.poscia,

I’ve made the changes that you suggested but the problem persists.

I needed to add these lines in my code to SonarQube to analyze the right commit.

git fetch origin pull/$PR/head:sonar/$PR
git status
git checkout sonar/$PR

My complete code is this way:

GIT_USER=my-git-user
GIT_TOKEN=my-git-token

rm -rf xxx

git clone https://$GIT_USER:$GIT_TOKEN@github.com/xxx.git

SONAR_HOST=my-sonarqube-address
SONAR_LOGIN=my-sonarqube-login
SONAR_PASSWORD=my-sonarqube-password
PR=pr-number
PR_BRANCH=pr-branch-name
BASE_BRANCH=main-branch

cd xxx

git fetch origin pull/$PR/head:sonar/$PR
git status
git checkout sonar/$PR

pwd
git status -b

JAVA_HOME=/opt/jdk-17
export MAVEN_OPTS="-Xms512m -Xmx4g"

# run the analysis
mvn clean verify sonar:sonar -Dmaven.test.failure.ignore=true \
-Dsonar.java.jdkHome=/opt/jdk-17 \
-DskipUTs=true \
-Dsonar.host.url=$SONAR_HOST \
-Dsonar.login=$SONAR_LOGIN \
-Dsonar.password=$SONAR_PASSWORD \
-Dsonar.verbose=true \
-Dsonar.pullrequest.key=$PR \
-Dsonar.pullrequest.branch=$PR_BRANCH \
-Dsonar.pullrequest.base=$BASE_BRANCH

I’ve checked and there is the .git folder in my project and my git version is git version 2.31.1.

One thing that I noticed is these warnings:

  • “SCM provider autodetection failed. Please use “sonar.scm.provider” to define SCM of your project, or disable the SCM Sensor in the project settings.”

  • Pull request decoration failed. Commit ‘000000000000000000000’ not found in pull request ‘8620’ (In this case the decoration shows in the Github but with another commit number)

Hi @flima33,

Thanks for trying, but could you send me the corresponding output?

Hi @aurelien.poscia

Follow the log from the last analysis:

build 06-Oct-2022 16:24:20 PR NUMBER FINAL:
build 06-Oct-2022 16:24:20 8620

build 06-Oct-2022 16:24:20 PR BRANCH FINAL:
build 06-Oct-2022 16:24:20 sonarqube-new-test

build 06-Oct-2022 16:24:21 On branch master
build 06-Oct-2022 16:24:21 Your branch is up to date with ‘origin/master’.
build 06-Oct-2022 16:24:21
build 06-Oct-2022 16:24:21 nothing to commit, working tree clean
error 06-Oct-2022 16:24:21 Switched to branch ‘sonar/8620’
build 06-Oct-2022 16:24:21 Log final:
build 06-Oct-2022 16:24:21 commit d9b6e53ba74f35a512c45709a00661c31bf06bf9

build 06-Oct-2022 16:24:21 GIT BRANCH STATUS
build 06-Oct-2022 16:24:21 On branch sonar/8620
build 06-Oct-2022 16:24:21 nothing to commit, working tree clean

build 06-Oct-2022 16:30:26 [INFO] User cache: /home/user/.sonar/cache
build 06-Oct-2022 16:30:26 [INFO] SonarQube version: 9.6.1.59531
build 06-Oct-2022 16:30:26 [INFO] Default locale: “en”, source code encoding: “UTF-8”
build 06-Oct-2022 16:30:27 [INFO] Load global settings
build 06-Oct-2022 16:30:27 [INFO] Load global settings (done) | time=235ms
build 06-Oct-2022 16:30:27 [INFO] Server id: xxxxxxxxx
build 06-Oct-2022 16:30:27 [INFO] User cache: /home/user/.sonar/cache
build 06-Oct-2022 16:30:27 [INFO] Load/download plugins
build 06-Oct-2022 16:30:27 [INFO] Load plugins index
build 06-Oct-2022 16:30:27 [INFO] Load plugins index (done) | time=58ms
build 06-Oct-2022 16:30:30 [INFO] Load/download plugins (done) | time=2659ms
build 06-Oct-2022 16:30:30 [INFO] Loaded core extensions: developer-scanner
build 06-Oct-2022 16:30:30 [INFO] Process project properties
build 06-Oct-2022 16:30:30 [INFO] Process project properties (done) | time=92ms
build 06-Oct-2022 16:30:30 [INFO] Execute project builders
build 06-Oct-2022 16:30:30 [INFO] Execute project builders (done) | time=1ms
build 06-Oct-2022 16:30:30 [INFO] Project key: xxxxxxx
build 06-Oct-2022 16:30:30 [INFO] Base dir: /home/user/myproject
build 06-Oct-2022 16:30:30 [INFO] Working dir: /home/user/myproject/project/target/sonar
build 06-Oct-2022 16:30:30 [INFO] Load project settings for component key: ‘xxxxxx’
build 06-Oct-2022 16:30:30 [INFO] Load project settings for component key: ‘xxxxxx’ (done) | time=36ms
build 06-Oct-2022 16:30:31 [INFO] Load project branches
build 06-Oct-2022 16:30:31 [INFO] Load project branches (done) | time=81ms
build 06-Oct-2022 16:30:31 [INFO] Load project pull requests
build 06-Oct-2022 16:30:31 [INFO] Load project pull requests (done) | time=43ms
build 06-Oct-2022 16:30:31 [INFO] Load branch configuration
build 06-Oct-2022 16:30:31 [INFO] Found manual configuration of branch/PR analysis. Skipping automatic configuration.
build 06-Oct-2022 16:30:31 [INFO] Load branch configuration (done) | time=2ms
build 06-Oct-2022 16:30:31 [WARNING] SCM provider autodetection failed. Please use “sonar.scm.provider” to define SCM of your project, or disable the SCM Sensor in the project settings.
build 06-Oct-2022 16:30:31 [INFO] Load quality profiles
build 06-Oct-2022 16:30:31 [INFO] Load quality profiles (done) | time=61ms
build 06-Oct-2022 16:30:31 [INFO] Load active rules
build 06-Oct-2022 16:30:33 [INFO] Load active rules (done) | time=1749ms
build 06-Oct-2022 16:30:33 [INFO] Load analysis cache
build 06-Oct-2022 16:30:34 [INFO] Load analysis cache | time=1652ms
build 06-Oct-2022 16:30:34 [INFO] Pull request 8620 for merge into master from sonarqube-new-test
build 06-Oct-2022 16:30:34 [INFO] Load project repositories
build 06-Oct-2022 16:30:34 [INFO] Load project repositories (done) | time=196ms
build 06-Oct-2022 16:30:35 [INFO] Indexing files…
build 06-Oct-2022 16:30:35 [INFO] Project configuration:

build 06-Oct-2022 16:30:43 [INFO] 20643 files indexed
build 06-Oct-2022 16:30:43 [INFO] 41 files ignored because of inclusion/exclusion patterns
build 06-Oct-2022 16:30:43 [INFO] Quality profile for css: Sonar way
build 06-Oct-2022 16:30:43 [INFO] Quality profile for java: Sonar way
build 06-Oct-2022 16:30:43 [INFO] Quality profile for js: Sonar way
build 06-Oct-2022 16:30:43 [INFO] Quality profile for json: Sonar way
build 06-Oct-2022 16:30:43 [INFO] Quality profile for jsp: Sonar way
build 06-Oct-2022 16:30:43 [INFO] Quality profile for web: Sonar way
build 06-Oct-2022 16:30:43 [INFO] Quality profile for xml: Sonar way

build 06-Oct-2022 16:45:56 [INFO] CPD Executor CPD calculation finished (done) | time=3048ms
build 06-Oct-2022 16:45:57 [INFO] Analysis report generated in 893ms, dir size=113.2 MB
build 06-Oct-2022 16:46:11 [INFO] Analysis report compressed in 13651ms, zip size=46.1 MB
build 06-Oct-2022 16:46:11 [INFO] Analysis report generated in /home/user/myproject/project/target/sonar/scanner-report
build 06-Oct-2022 16:46:13 [INFO] Analysis report uploaded in 1966ms
build 06-Oct-2022 16:46:13 [INFO] ANALYSIS SUCCESSFUL, you can find the results at: https://url.com/dashboard?id=myproject%3Aproject&pullRequest=8620
build 06-Oct-2022 16:46:13 [INFO] Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
build 06-Oct-2022 16:46:13 [INFO] More about the report processing at https://url.com/api/ce/task?id=xxxxxxx
build 06-Oct-2022 16:46:13 [INFO] Analysis total time: 15:43.186 s

Hi @aurelien.poscia,

Were you able to review the logs?

Thanks,

Yes and I also tried to reproduce locally, without sucess.

SonarQube relies on JGIT to identify that a directory is GIT-managed. Here is what JGit checks:

  1. /home/user/myproject/.git directory exists
  2. /home/user/myproject/objects exists
  3. /home/user/myproject/refs exists
  4. /home/user/myproject/reftable exists OR (/home/user/myproject/HEAD exists AND startwith ref: refs/)

Could you

  1. verify that all those files exists?
  2. tell me if you are using a multi-module project?
  3. ideally, come up with a minimal reproducer project that I can run myself
  4. send me the logs with --debug flag?