Integration build with SonarQube analysis passes when it should fail

SonarScanner v4.11.0

Hi all!

I’ve written about this issue before, but I made various incorrect assumptions, and didn’t understand exactly what caused the issue - but please still bear with me as I am new to SonarQube!

So we basically have an Azure DevOps Build pipeline setup (it doesn’t make a difference that this is in an Azure DevOps pipeline, the same thing would happen if I ran all of this manually on my PC) - this pipeline simply does a ‘Prepare Analysis Configuration’ step (this is a step designed by SonarQube for integration with the pipeline), then it builds the solution, runs the unit tests, and then finalizes the code analysis.

This configuration can be seen below:

image
The analysis is set to integrate with MSBuild.
The analysis itself works as intended - we don’t have any issues there. The issue is that there’s 1 edge case we’ve found so far where it doesn’t work as intended.

If we run this build with an unused variable reference, it should fail the build, as this is how we have it configured - but for some reason, the build passes successfully, and instead reports the issue as a code smell.
If we run this same version of the code with the same unused variable reference, but without the SonarQube analysis steps before and after the build, the build fails as we would expect with (approximately) the following error:

Source\SomeCodeProbably\NormalViewModel.cs(230,22): Error CS0414: The field ‘NormalViewModel.disposed’ is assigned but its value is never used

Does anyone have any idea why SonarQube does this, and how we can make it work as expected?

With other issues that SonarQube picks up as code smells, if we run the non-SonarQube build on that version of the code, it passes successfully - as we would expect for those code-smells, there just seems to be this one point of overlap between SonarQube and our build rules where SonarQube seems to catch the build error and instead report it as a code smell!

Help is appreciated, and apologies if I’ve missed any information, I’ll happily add more information!

Hi @Red_Blob,

This seems duplicate to another thread that I’ve just replied to:

This looks like the same thing. Basically, you have the problem when SonarQube is not present. So it’s not SonarQube related :slight_smile:

Hi!

Thanks for the response. :slight_smile:

While that is the same issue, sadly the fix mentioned isn’t the way forward.

I think you may misunderstand - we want the build to fail with this error, this is the correct behaviour. The problem is that SonarQube seems to swallow this error, rather than fail the build.

So I’m looking to figure out why the SonarQube analysis step causes the MSBuild step to no longer fail as I would expect.

Thanks!

Hi!

Thank you for the clarification. Can you tell me what exact settings to you have in your project (probably in csproj or targets file) to make the build fail (without SQ) so I can reproduce it locally?

Hi,

So I’m not 100% sure which properties of the CSPROJ file would be useful for reproducing the issue, but I think our build error settings should be enough:

Nothing else is configured ‘abnormally’, and as far as I can tell, the build error is a default MSBuild compiler warning that can be an error if configured with these error settings.

Thanks! :slight_smile:

Hi @Red_Blob,

I’m able to reproduce your issue with TreatWarningsAsErrors set to true.

How it works

Compiler message CS0414 is a warning, just as any other warning like S1481 from our analyzer for C#.
Using this

<TreatWarningsAsErrors>true</TreatWarningsAsErrors>

in your projects changes all warnings to errors. That setting would fail your build for every little code smell or false positive that is raised by our Analyzer. And you would never see your results in SonarQube unless you clear them all.

To prevent this, Scanner for MsBuild injects it’s own .targets file in the build that you can find in .sonarqube\bin\targets\SonarQube.Integration.targets Among other things, this file sets

<TreatWarningsAsErrors>false</TreatWarningsAsErrors>
<WarningsAsErrors></WarningsAsErrors>

in order to

  • prevent build from failing
  • display analysis results in SonarQube

What you should do

You should not fail your build on compiler warnings. You should rather import them in SQ and configure your Quality Profile to fail your Quality Gate when such issues exists. You can also keep the project settings as is to fail your build locally, if needed for faster development loop.

There are several options to configure when importing external issues.

What you can not do

You can not set TreatWarningsAsErrors to true for SQ analysis. That would fail every build with any reported issue.

What you can do

You can keep the TreatWarningsAsErrors in your configuration to fail locally and override the WarningsAsErrors property to fail when SQ analysis is run. You’ll need to list all CS compiler warnings that you want to fail your build.

Do inject this settings, you’ll need to update your .targets file or create new one (e.g. Directory.Build.targets in your project root directory) with content like this:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <Target Name="SonarQubeRestoreWarningsAsErrors"
        Condition=" $(SonarQubeTempPath) != '' "
        AfterTargets="OverrideRoslynCodeAnalysisProperties"
        BeforeTargets="CoreCompile">
    <PropertyGroup>
      <WarningsAsErrors>CS0414</WarningsAsErrors>
    </PropertyGroup>
  </Target>
</Project>

You can find more details about CSC.WarningAsErrors in Microsoft docs.

Pavel