How to reduce build time while using sonar

SonarQube Cloud
,
1
  • GitHub
  • AWS CodeBuild
  • C# (dotnet 5)

We have a CI/CD pipeline in AWS, our solution has 50 csproj, this solution has 21 AWS lambdas and the rest are shared projects used by the lambdas.

Our problem is that the build is taking about 18 minutes, and most of that time is sonar analyzing the same projects for each of our lambdas, the structure of our buildspec.yml is something like this:

 - dotnet tool install --global dotnet-sonarscanner --version 5.2
    - dotnet restore --disable-parallel --verbosity m src/MySolution.sln --configfile src/.Nuget/NuGet.config /property:Configuration=Release
	 - >-
        dotnet sonarscanner begin
        /k:"something"
        /n:"something"
        /v:"$API_VERSION"
        /o:"something"
        /d:"sonar.cs.vstest.reportsPaths=src/test-results/something.trx"
        /d:"sonar.cs.opencover.reportsPaths=src/test-results/**/coverage.opencover.xml"
        /d:"sonar.host.url=https://sonarcloud.io"
        /d:"sonar.login=$SONAR_TOKEN"
        /d:"sonar.sources=src"
        /d:"sonar.language=cs"
        /d:"sonar.branch.name=$BRANCHNAME"
        /d:"sonar.sourceEncoding=UTF-8"
      
	- dotnet publish --no-restore -c Release -o Outout src/Code/Lambda.csproj
	-- repeat this 20 times for each lambda
    
	 - >-
        dotnet test
        --no-restore
        --configuration Release
        --logger "trx;LogFileName=something.trx"
        --collect:"XPlat Code Coverage"
        --results-directory src/test-results
        /p:CollectCoverage=true
        /p:ExcludeByAttribute=ExcludeFromCodeCoverage
        /p:Threshold=100
        /p:ThresholdType=\"Method,Line,Branch\"
        --settings runsettings.xml
        src/Tests/UnitTests.csproj
    - >-
        dotnet sonarscanner end
        /d:"sonar.login=$SONAR_TOKEN"

We googled and found a parameter to exclude commands from sonar, but after the change the code coverage is 0%, we modified the buildspec to something like this:

- dotnet tool install --global dotnet-sonarscanner --version 5.2
	- dotnet restore --disable-parallel --verbosity m src/MySolution.sln --configfile src/.Nuget/NuGet.config /property:Configuration=Release
    - >-
        dotnet sonarscanner begin
        /k:"something"
        /n:"something"
        /v:"$API_VERSION"
        /o:"something"
        /d:"sonar.cs.vstest.reportsPaths=src/test-results/something.trx"
        /d:"sonar.cs.opencover.reportsPaths=src/test-results/**/coverage.opencover.xml"
        /d:"sonar.host.url=https://sonarcloud.io"
        /d:"sonar.login=$SONAR_TOKEN"
        /d:"sonar.sources=src"
        /d:"sonar.language=cs"
        /d:"sonar.branch.name=$BRANCHNAME"
        /d:"sonar.sourceEncoding=UTF-8"
        	
	- dotnet build --no-restore -c Release -o Discard src/MySolution.sln	
	
	- >-
        dotnet test
        --no-restore
        --configuration Release
        --logger "trx;LogFileName=something.trx"
        --collect:"XPlat Code Coverage"
        --results-directory src/test-results
        /p:CollectCoverage=true
        /p:ExcludeByAttribute=ExcludeFromCodeCoverage
        /p:Threshold=100
        /p:ThresholdType=\"Method,Line,Branch\"
        --settings runsettings.xml
        src/Tests/UnitTests.csproj
    - >-
        dotnet sonarscanner end
        /d:"sonar.login=$SONAR_TOKEN"
		
    - dotnet publish --no-restore -p:SonarQubeTargetsImported=true -c Release -o Output src/Lambda.csproj
	-- repeat this 20 times for each lambda

What are we missing to avoid sonar to analyze multiple times the projects and have code coverage of our unit tests?

Thanks for your time.
Juan Zamudio

2

Hi @JuanZamudioGBM - welcome to the community!

Analysis is done as part of the build, and both dotnet publish and dotnet test will build unless you specify --no-build. Have you tried passing that argument?

Separately, you shouldn’t pass sonar.sources when using the Scanner for .Net, and sonar.language has been deprecated for some time.

3

Hi @duncanp
We tried to add --no-build as suggested, that reduced the build time significantly, but now the code coverage in sonarcloud is 0. We are using .net core 3.1 (I incorrectly said it was 5) and coverlet.collector 3.0.3 for the code coverage, our buildspec now is something like this (I still dont remove the parts that you said are no longer necessary):

- exec /usr/local/bin/dotnet-install.sh -v 3.1.408
    - dotnet tool install --global dotnet-sonarscanner --version 5.2
    - dotnet restore --disable-parallel --verbosity m src/Mysolution.sln --configfile src/.Nuget/NuGet.config /property:Configuration=Release
	- >-
        dotnet sonarscanner begin
        /k:"something"
        /n:"something"
        /v:"$API_VERSION"
        /o:"something"
        /d:"sonar.cs.vstest.reportsPaths=src/test-results/UnitTests.trx"
        /d:"sonar.cs.opencover.reportsPaths=src/test-results/**/coverage.opencover.xml"
        /d:"sonar.host.url=https://sonarcloud.io"
        /d:"sonar.login=$SONAR_TOKEN"
        /d:"sonar.sources=src"
        /d:"sonar.language=cs"
        /d:"sonar.branch.name=$BRANCHNAME"
        /d:"sonar.sourceEncoding=UTF-8"
        
    - dotnet build --no-restore -c Release src/Mysolution.sln
    - dotnet publish --no-restore --no-build -c Release -o Output src/MyProject.csproj
    -- repeated this for other lambdas
    - >-
        dotnet test
        --no-restore
        --no-build
        --configuration Release
        --logger "trx;LogFileName=UnitTests.trx"
        --collect:"XPlat Code Coverage"
        --results-directory src/test-results
        /p:CollectCoverage=true
        /p:ExcludeByAttribute=ExcludeFromCodeCoverage
        /p:Threshold=100
        /p:ThresholdType=\"Method,Line,Branch\"
        --settings runsettings.xml
        src/Tests/UnitTests.csproj
    - >-
        dotnet sonarscanner end
        /d:"sonar.login=$SONAR_TOKEN"

Thanks for your time

4

Hi @JuanZamudioGBM,

Is the test project referenced by the solution file?

Does the coverage appear if you remove the --no-build from the dotnet test` command?

5

Is the test project referenced by the solution file?
Yes

Does the coverage appear if you remove the --no-build from the dotnet test` command?
We tried with and without the parameter and no coverage was generated

Thanks

6

Any other thing I could try? The only way to generate the coverage in sonar cloud is the remove the --no-build flag fro the publish commands but we are close to the timeout in our pipeline and we need to add more lambdas to the solution.

Thanks for your time

9

Hi

We recently published a guide to investigate performance issue, you can find it here: The Sonar guide for investigating the performance of .NET analysis .

Some ideas:

  • looking at the detailed logs to work out where the test coverage that is imported is being generated so they can work out which steps they really need to run
  • exclude the shared projects from this analysis and analyze them separately in a separate pipeline with a separate SC project - it is not optimal, however, like this they will only be analyzed once

Please check out the guide and come back with any additional questions.