.Net Scanner not Finishing scan

SonarQube Cloud
, ,
1

Template for a good new topic, formatted with Markdown:

  • ALM used: Github
  • CI system used: Github
  • Scanner command used when applicable:
./.sonar/scanner/dotnet-sonarscanner begin /k:"RegScale_regscale" /o:"regscale" /d:sonar.login="***" /d:sonar.host.url="https://sonarcloud.io" \
     \
    /d:sonar.javascript.node.maxspace=8192 \
    /d:sonar.exclusions=Atlas.Entities/Migrations/**,Atlas.Angular/node_modules/**
  dotnet build ./atlas/Atlas.csproj # path is important
  ./.sonar/scanner/dotnet-sonarscanner end /d:sonar.login="***" | tee /home/runner/sonar-scan-output.txt
  # Get the sonarqube task url from the sonar scan output
  task_url=$(grep "More about the report processing at" /home/runner/sonar-scan-output.txt | awk '{print $NF}')
  # Add the sonar API authentication token to the url.
  echo task_url=${task_url/sonarcloud.io/***@sonarcloud.io} >> "$GITHUB_ENV"
  • Languages of the repository: C#, XML, CSS
  • Error observed (wrap logs/code around with triple quotes ``` for proper formatting):
    • No Error produced. Scan does not finish the end step on dotnet-sonarscanner 6.1. On dotnet-sonarscanner 7.1.1 we get a memory out of Range error on the end step before it does anything
2

Hi,

Welcome to the community!

Please share your analysis log.

Share the Scanner for .NET verbose logs

  • Add /d:"sonar.verbose=true" to the…
    • SonarScanner.MSBuild.exe or dotnet sonarscanner begin command to get more detailed logs
      • For example: SonarScanner.MSBuild.exe begin /k:"MyProject" /d:"sonar.verbose=true"
    • “SonarQubePrepare” or “SonarCloudPrepare” task’s extraProperties argument if you are using Azure DevOps
      • For example:
        - task: SonarCloudPrepare@1
    inputs:
      SonarCloud: 'sonarcloud'
      organization: 'foo'
      scannerMode: 'MSBuild'
      projectKey: 'foo_sonar-scanning-someconsoleapp'
      projectName: 'sonar-scanning-someconsoleapp'
      extraProperties: |
        sonar.verbose=true
  • The important logs are in the END step (i.e. SonarQubeAnalyze / SonarCloudAnalyze / “Run Code Analysis”)

Share the msbuild detailed logs

MsBuild.exe /t:Rebuild /v:d

or

dotnet build -v:d

 
Thx,
Ann

3

I am including the log (super long), with the verbose flag on it as well as the build and end logs. The end log is next to useless since it has the issue I stated before where it doesn’t throw an error and just doesn’t finish. It also doesn’t post the results in our sonar cloud instance because it doesn’t finish.

The end logs are as follows:

2024-08-07T15:54:57.7296965Z SonarScanner for MSBuild 6.1
2024-08-07T15:54:57.7297479Z Using the .NET Core version of the Scanner for MSBuild
2024-08-07T15:54:57.7297747Z Post-processing started.
2024-08-07T15:54:57.7298518Z 15:54:20.001  15:54:19.998  Uninstalling target: /home/runner/work/regscale/regscale/.sonarqube/bin/targets/SonarQube.Integration.targets
2024-08-07T15:54:57.7299276Z 15:54:20.001  15:54:20.001  sonar.verbose=true was specified - setting the log verbosity to 'Debug'
2024-08-07T15:54:57.7300092Z 15:54:20.002  Loading the SonarCloud analysis config from /home/runner/work/regscale/regscale/.sonarqube/conf/SonarQubeAnalysisConfig.xml
2024-08-07T15:54:57.7300421Z 15:54:20.002  Not running under TeamBuild
2024-08-07T15:54:57.7300999Z 15:54:20.002  Analysis base directory: /home/runner/work/regscale/regscale/.sonarqube
2024-08-07T15:54:57.7301157Z Build directory: 
2024-08-07T15:54:57.7301555Z Bin directory: /home/runner/work/regscale/regscale/.sonarqube/bin
2024-08-07T15:54:57.7301979Z Config directory: /home/runner/work/regscale/regscale/.sonarqube/conf
2024-08-07T15:54:57.7302441Z Output directory: /home/runner/work/regscale/regscale/.sonarqube/out
2024-08-07T15:54:57.7303074Z Config file: /home/runner/work/regscale/regscale/.sonarqube/conf/SonarQubeAnalysisConfig.xml
2024-08-07T15:54:57.7303971Z 15:54:20.003  Generating SonarCloud project properties file to /home/runner/work/regscale/regscale/.sonarqube/out/sonar-project.properties
2024-08-07T15:54:57.7304419Z 15:54:20.021  Setting analysis property: sonar.visualstudio.enable=false

sonar_run_08-07-24.zip (3.2 MB)

4

Hi,

Thanks for the log. It’s actually pretty helpful. The very end is the relevant part:

2024-08-07T15:54:57.7303074Z Config file: /home/runner/work/regscale/regscale/.sonarqube/conf/SonarQubeAnalysisConfig.xml
2024-08-07T15:54:57.7303971Z 15:54:20.003  Generating SonarCloud project properties file to /home/runner/work/regscale/regscale/.sonarqube/out/sonar-project.properties
2024-08-07T15:54:57.7304419Z 15:54:20.021  Setting analysis property: sonar.visualstudio.enable=false
2024-08-07T15:54:57.7355261Z ##[group]Run echo Checking task status at API url: 
2024-08-07T15:54:57.7355683Z e[36;1mecho Checking task status at API url: e[0m
2024-08-07T15:54:57.7355838Z e[36;1mfor i in {1..20}; doe[0m
2024-08-07T15:54:57.7356328Z e[36;1m  curl -m 10 -s "" | grep '"status":"SUCCESS"' && break || echo "Waiting for analysis to complete..."; sleep 10;e[0m
2024-08-07T15:54:57.7356478Z e[36;1mdonee[0m
2024-08-07T15:54:57.7391313Z shell: /usr/bin/bash -e {0}
2024-08-07T15:54:57.7391438Z env:
2024-08-07T15:54:57.7391855Z   IMAGE_NAME_ACR: regscalecontainerregistry.azurecr.io/regscale-internal
2024-08-07T15:54:57.7392239Z   JAVA_HOME: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.12-7/x64
2024-08-07T15:54:57.7392501Z   JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.12-7/x64
2024-08-07T15:54:57.7392688Z   task_url: 
2024-08-07T15:54:57.7392816Z ##[endgroup]
2024-08-07T15:54:57.7453912Z Checking task status at API url:
2024-08-07T15:54:57.7516326Z Waiting for analysis to complete...
2024-08-07T15:55:07.7601372Z Waiting for analysis to complete...
2024-08-07T15:55:17.7678484Z Waiting for analysis to complete...
2024-08-07T15:55:27.7758584Z Waiting for analysis to complete...
2024-08-07T15:55:37.7836296Z Waiting for analysis to complete...
2024-08-07T15:55:47.7913996Z Waiting for analysis to complete...
2024-08-07T15:55:57.7996001Z Waiting for analysis to complete...
2024-08-07T15:56:07.8075074Z Waiting for analysis to complete...
2024-08-07T15:56:17.8153250Z Waiting for analysis to complete...
2024-08-07T15:56:27.8232431Z Waiting for analysis to complete...
2024-08-07T15:56:37.8311527Z Waiting for analysis to complete...
2024-08-07T15:56:47.8391633Z Waiting for analysis to complete...
2024-08-07T15:56:57.8469467Z Waiting for analysis to complete...
2024-08-07T15:57:07.8550827Z Waiting for analysis to complete...
2024-08-07T15:57:17.8629875Z Waiting for analysis to complete...
2024-08-07T15:57:27.8709772Z Waiting for analysis to complete...
2024-08-07T15:57:37.8789935Z Waiting for analysis to complete...
2024-08-07T15:57:47.8867655Z Waiting for analysis to complete...
2024-08-07T15:57:57.8948866Z Waiting for analysis to complete...
2024-08-07T15:58:07.9028585Z Waiting for analysis to complete...

Note that that “Waiting for analysis to complete…” message comes from the script at the end:

2024-08-07T15:54:57.7355683Z e[36;1mecho Checking task status at API url: e[0m
2024-08-07T15:54:57.7355838Z e[36;1mfor i in {1..20}; doe[0m
2024-08-07T15:54:57.7356328Z e[36;1m  curl -m 10 -s "" | grep '"status":"SUCCESS"' && break || echo "Waiting for analysis to complete..."; sleep 10;e[0m
2024-08-07T15:54:57.7356478Z e[36;1mdonee[0m
2024-08-07T15:54:57.7391313Z shell: /usr/bin/bash -e {0}

That said, there are things I don’t see in the end step that I expect to, including the generation, compression & transmission of the analysis report.

Can you post your pipeline?

 
Thx,
Ann

5

I am unable to post the pipeline. However that is the issue. The Checking task status at API URL: is an echo statement that is supposed to paste the URL that it receives from the output statement from the sonar end step. Since the end step never finishes it never gets to the URL that is needed for the evaluation you are referring to.

The file I uploaded was the log for the entire job run minus the checkout steps.

6

Hi,

I’m not so sure about that.

Here’s what we appear to have in the log from the end step:

2024-08-07T15:54:57.7296965Z SonarScanner for MSBuild 6.1
2024-08-07T15:54:57.7297479Z Using the .NET Core version of the Scanner for MSBuild
2024-08-07T15:54:57.7297747Z Post-processing started.
2024-08-07T15:54:57.7298518Z 15:54:20.001  15:54:19.998  Uninstalling target: /home/runner/work/regscale/regscale/.sonarqube/bin/targets/SonarQube.Integration.targets
2024-08-07T15:54:57.7299276Z 15:54:20.001  15:54:20.001  sonar.verbose=true was specified - setting the log verbosity to 'Debug'
2024-08-07T15:54:57.7300092Z 15:54:20.002  Loading the SonarCloud analysis config from /home/runner/work/regscale/regscale/.sonarqube/conf/SonarQubeAnalysisConfig.xml
2024-08-07T15:54:57.7300421Z 15:54:20.002  Not running under TeamBuild
2024-08-07T15:54:57.7300999Z 15:54:20.002  Analysis base directory: /home/runner/work/regscale/regscale/.sonarqube
2024-08-07T15:54:57.7301157Z Build directory: 
2024-08-07T15:54:57.7301555Z Bin directory: /home/runner/work/regscale/regscale/.sonarqube/bin
2024-08-07T15:54:57.7301979Z Config directory: /home/runner/work/regscale/regscale/.sonarqube/conf
2024-08-07T15:54:57.7302441Z Output directory: /home/runner/work/regscale/regscale/.sonarqube/out
2024-08-07T15:54:57.7303074Z Config file: /home/runner/work/regscale/regscale/.sonarqube/conf/SonarQubeAnalysisConfig.xml
2024-08-07T15:54:57.7303971Z 15:54:20.003  Generating SonarCloud project properties file to /home/runner/work/regscale/regscale/.sonarqube/out/sonar-project.properties
2024-08-07T15:54:57.7304419Z 15:54:20.021  Setting analysis property: sonar.visualstudio.enable=false

Then it goes straight into your polling script.

That’s not a normal end step log at all. At this point, my guess is that something is aborting the process. That’s why I wanted to see your pipeline.

 
Ann

7

Thats why I said it was not finishing. The logs show everything the pipeline does. I am not sure what could be aborting it. If you point me in a direction thats not logs that could tell me where to look for that I can. Previously It would upload to sonar cloud and produce a link (which is what the polling is looking for).

We are doing the setup, sonar begin, build, and then end is not finishing (or stopping). I even tried to do the sonar github action to see if that fixes things but the github action does not perform the build so nothing was actually scanned (may not have configured the action correctly for a .Net application).

It was working until about 5 months ago, and our pipeline was spitting out the green checkmark. It wasnt until we had to look back to see if we had cleared a finding that we noticed it hasnt been working this entire time

8

Hi,

What changed 5 months ago?

Honestly, I’m not sure where to tell you to look. It was working and it stopped. You’re going to have to look outside of analysis to find out why. I would start with any other processes on the host that might be killing the process.

 
HTH,
Ann

9

We build a different pipeline yml 5 months ago. However that job has not been changed and was copied and pasted from when it was first added over a year and a half ago. We are using github runners so we have no access to see if any other process is killing this process. Thats why I tried opening a support ticket and they directed me here.

10

Hi,

I know correlation is not causation, but this seems awfully suspicious. Can you go back to the old pipeline?

 
Ann

11

Unfortunately we are unable to go back to the original pipeline. We did an overhaul of it and changed branching strategies which needed the new pipeline. As I stated before I copied and pasted the job as is and have not made changes to it. The logs from 5 months ago are also unavailable so I cannot access those to see if there is a difference in the runners.

12

Hi,

Sorry, but I’m out of ideas.

 
Ann

13

Is there some way to escalate or get someone from support to look at this?

14

I think I may have found the issue. We are excluding the Atlas.Entities/Migrations/** Folder and all of its contents but if you look back at the file I uploaded they are listed (they are the majority of the 87K warnings found). I am not sure but I think my exclusions may not be happening which may be causing an out of memory exception since there are so many of them. Is there any way to actually stop them from scanning because it doesnt seem like the exclusion list is actually working.

Example of exclusions i have tried:

/d:sonar.exclusions=/home/**/Atlas.Entities/Migrations/**,/home/**/Atlas.Angular/node_modules/**

/d:sonar.exclusions=**/Atlas.Entities/Migrations/**,**/Atlas.Angular/node_modules/**

and
/d:sonar.exclusions=Atlas.Entities/Migrations/**,Atlas.Angular/node_modules/**

15

Hi,

:grimacing:

With all the other scanners, exclusions prevent the files from being analyzed. With SonarScanner for .NET, exclusions prevent the finding on the files from being uploaded.

 
Ann

16

So is there no way to truly exclude them from being analyzed in SonarScanner for .NET? I have even added it to the csproj file and it still does it. I think this is the issue but we need the project scanned just not those 1500+ files that are located in that folder. I have not seen anything in the documentation to help out with this but I do believe this is the issue as we have added a few hundred of these files in the past 6 months.

17

Hi,

Excluding projects from analysis is the only thing that comes to mind (assuming the files are grouped tidily for that). Although, TBH, I don’t know if the mechanism is a pre-exclusion, like you need, or another don’t-submit.

 
Ann

18

OK so we need the rest of the project to be scanned so that’s not an option. So because we are using the .NET scanner we cannot exclude like we need. Is there another way to scan a .NET project correctly without using the .NET scanner so that we can actually get this to work?

19

Hi,

Unfortunately, there is not.

Since you believe this is about the OOM, have you tried cranking the memory settings up to 11?

 
Ann

20

I have just tried increasing the sonar.javascript.node.maxspace a few times. It was set to 8gb initially. I have done 10, 16, 20, and 32GB now. All of them have now failed with Out of memory. Is there another memory setting I should be adding?