Adding project to .NET solution makes all code new

samd · December 20, 2021, 10:27pm

AzureDevops
C#/.NET 5

We have a .NET 5 repo setup as a single solution with multiple projects. Everything has been working fine for the last year or so, but we recently added a new project–an Azure Function. This has been added to our build pipeline yaml as follows (in between SonarCloud prepare and SonarCloud analyze):

- task: DotNetCoreCLI@2
  displayName: "Build Azure Function"
  inputs:
    command: build
    projects: 'azure.function/azure.function.csproj'
    arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)/function'

For some reason, however, once we add this task to the pipeline all of our previous code gets analyzed as if it were new (i.e. that branch is listed as having x thousand lines of new code and all our “old” code smells show up in the analysis).

The only hint I can see about why this is happening is that, in SonarCloud, on the “Code” screen for that branch it shows home/azure-agent/agent/_work/18/s as the directory where the new lines are (i.e., the temp directory used by our build agent) rather than just showing all the code directories at the root of the project. In other words, it looks like for some reason it think it is viewing our code in a separate “new” subdirectory rather than the root.

But we haven’t changed anything about our repo setup, and this project is setup to build/scan just like all the other projects in our repo. Any ideas what could be causing this?

samd · January 1, 2022, 5:56pm

Going through the logs of the “analyze” stage of the original yaml and the one with this new .csproj added, it looks like the problem comes from the following:

original (correct) yaml - Base dir: /home/azure-agent/agent/_work/18/s

new yaml - Base dir: /

I’m not manually setting the base directory, so what could be causing it to get different values? Put another way, what goes into the determinatino of base dir?

samd · January 1, 2022, 6:53pm

I’m able to work around this by adding the following to extraProperties in the devops pipeline yaml for the SonarCloudPrepare task:

sonar.projectBaseDir=$(Build.Repository.LocalPath)/

Open to suggestions for better ways to solve this.

Colin · January 3, 2022, 10:46am

Hey @samd

I’m not an expert here, but I can point to the logic for determining the base directory.

github.com

SonarSource/sonar-scanner-msbuild/blob/6bca2a2ee42c939c3396c907a5d62709fe6aa11d/src/SonarScanner.MSBuild.Shim/PropertiesFileGenerator.cs#L402


      
                      {
                          Id = reportFilesPropertyKey,
                          Value = string.Join(RoslynReportPathsDelimiter.ToString(), listOfPaths)
                      };
                      project.AnalysisSettings.Add(newReportPathProperty);
                  }
              }
          }
          
          /// <summary>
          /// Appends the sonar.projectBaseDir value. This is calculated as follows:
          /// 1. the user supplied value, or if none
          /// 2. the sources directory if running from TFS Build or XAML Build, or
          /// 3. the common root path of projects, or if there isn't any
          /// 4. the .sonarqube/out directory
          /// </summary>
          public DirectoryInfo ComputeRootProjectBaseDir(IEnumerable<DirectoryInfo> projectPaths)
          {
              DirectoryInfo rootDirectory;
          
              var projectBaseDir = analysisConfig.LocalSettings

Does your Azure Function exist far outside of the rest of your projects (with respect to the filesystem)?
Just to confirm, are you using Git or TFVC? I ask because even with the base directory changing, I would have expected file move detection to help out here.

samd · January 3, 2022, 3:14pm

No; that’s what’s weird. The file structure for the function is identical to all the other projects. A folder at the root of the directory that contains the .sln file.
Git.

Colin · January 6, 2022, 2:22pm

Hey @samd

Thanks for the details.

So just to confirm – previously you had a single DotNetCoreCLI@2 task building one solution sandwiched by the SonarCloud tasks, and now you have two DotNetCoreCLI@2 tasks (adding the second which builds the azure function). Is that correct? Just want to make sure I have the right conditions to try and reproduce.

samd · January 6, 2022, 2:46pm

So, that’s the right concept, but it is slightly more complex. We had three dot net projects sandwiched, and it was adding the fourth that caused this problem. The only thing different about the new project is the name of the directory it is in, but the structure and location in our directory structure is the same.

Another wrinkle is we also had a unit test project running in the middle and if we added a new unit test project to that task, it would also cause the same issue (even if we didn’t add the extra build project). But the issue would still happen even if i completely removed the unit tests from the “sandwich” and just added the additional build project.

This is a proprietary project so I didn’t post the yaml here, but if you would like me to message it to you somehow, I don’t have any issue with that.

Thank you so much for your time on this issue. We are working around it fine by manually specifying the build directory, but I’m more than happy to help continue to troubleshoot it. Thanks again!

Colin · January 6, 2022, 3:33pm

Thanks @samd! One more ask.

You can see from the code I referenced earlier that there’s some DEBUG level logging available to us about how the base directory is calculated.

Setting the analysis parameter sonar.verbose=true should reveal to us how the base directory is being derived. See here (relevant line surrounded by asteriks):

SonarScanner for MSBuild 5.4.1
Using the .NET Core version of the Scanner for MSBuild
Post-processing started.
16:31:24.058  16:31:24.031  Uninstalling target: /Users/colin/Source/dotnet-test-base/dotnet-test/.sonarqube/bin/targets/SonarQube.Integration.targets
16:31:24.058  16:31:24.058  sonar.verbose=true was specified - setting the log verbosity to 'Debug'
16:31:24.059  Loading the SonarCloud analysis config from /Users/colin/Source/dotnet-test-base/dotnet-test/.sonarqube/conf/SonarQubeAnalysisConfig.xml
16:31:24.059  Not running under TeamBuild
16:31:24.059  Analysis base directory: /Users/colin/Source/dotnet-test-base/dotnet-test/.sonarqube
Build directory: 
Bin directory: /Users/colin/Source/dotnet-test-base/dotnet-test/.sonarqube/bin
Config directory: /Users/colin/Source/dotnet-test-base/dotnet-test/.sonarqube/conf
Output directory: /Users/colin/Source/dotnet-test-base/dotnet-test/.sonarqube/out
Config file: /Users/colin/Source/dotnet-test-base/dotnet-test/.sonarqube/conf/SonarQubeAnalysisConfig.xml
16:31:24.06  Generating SonarCloud project properties file to /Users/colin/Source/dotnet-test-base/dotnet-test/.sonarqube/out/sonar-project.properties
16:31:24.089  Setting analysis property: sonar.visualstudio.enable=false
16:31:24.106  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /Users/colin/Source/dotnet-test-base/dotnet-test/.sonarqube/out/0/Issues.json
**16:31:24.123  Using longest common projects root path as project base directory: '/Users/colin/Source/dotnet-test-base/dotnet-test'.**
16:31:24.135  Dumping content of sonar-project.properties

This would be useful information. It would also be useful to know what method is being used to derive the base directory when you revert to a build without this new project (if it’s feasible as a test).

And finally (the big guns), if you’re willing to share the unredacted yaml and output of the End step when running in verbose, I’ll be happy to raise a private message with you.

samd · January 6, 2022, 4:05pm

Ok. I’m not sure this tells us that much. The both say they are using longest common projects root path.

Here’s the original (without new project):

2022-01-06T15:58:19.7830061Z ==============================================================================
2022-01-06T15:58:19.9356435Z [command]/home/azure-agent/agent/_work/_tool/dotnet/dotnet /home/azure-agent/agent/_work/_tasks/SonarCloudPrepare_14d9cde6-c1da-4d55-aa01-2965cd301255/1.23.1/dotnet-sonar-scanner-msbuild/SonarScanner.MSBuild.dll end
2022-01-06T15:58:20.0105643Z SonarScanner for MSBuild 5.4
2022-01-06T15:58:20.0107789Z Using the .NET Core version of the Scanner for MSBuild
2022-01-06T15:58:20.0783626Z Post-processing started.
2022-01-06T15:58:20.1504363Z 15:58:20.15  15:58:20.115  Uninstalling target: /home/azure-agent/agent/_work/18/.sonarqube/bin/targets/SonarQube.Integration.targets
2022-01-06T15:58:20.1505678Z 15:58:20.15  15:58:20.149  sonar.verbose=true was specified - setting the log verbosity to 'Debug'
2022-01-06T15:58:20.1509222Z 15:58:20.15  Loading the SonarCloud analysis config from /home/azure-agent/agent/_work/18/.sonarqube/conf/SonarQubeAnalysisConfig.xml
2022-01-06T15:58:20.1510029Z 15:58:20.15  Not running under TeamBuild
2022-01-06T15:58:20.1512536Z 15:58:20.151  Analysis base directory: /home/azure-agent/agent/_work/18/.sonarqube
2022-01-06T15:58:20.1513019Z Build directory: 
2022-01-06T15:58:20.1513622Z Bin directory: /home/azure-agent/agent/_work/18/.sonarqube/bin
2022-01-06T15:58:20.1514419Z Config directory: /home/azure-agent/agent/_work/18/.sonarqube/conf
2022-01-06T15:58:20.1515378Z Output directory: /home/azure-agent/agent/_work/18/.sonarqube/out
2022-01-06T15:58:20.1516346Z Config file: /home/azure-agent/agent/_work/18/.sonarqube/conf/SonarQubeAnalysisConfig.xml
2022-01-06T15:58:20.1530978Z 15:58:20.152  Generating SonarCloud project properties file to /home/azure-agent/agent/_work/18/.sonarqube/out/sonar-project.properties
2022-01-06T15:58:20.1800245Z 15:58:20.179  Setting analysis property: sonar.visualstudio.enable=false
2022-01-06T15:58:20.1897859Z 15:58:20.189  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/2/Issues.json
2022-01-06T15:58:20.1906262Z 15:58:20.19  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/3/Issues.json
2022-01-06T15:58:20.1909357Z 15:58:20.19  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/5/Issues.json
2022-01-06T15:58:20.1910832Z 15:58:20.19  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/6/Issues.json
2022-01-06T15:58:20.1921607Z 15:58:20.191  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/4/Issues.json
2022-01-06T15:58:20.1923081Z 15:58:20.191  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/0/Issues.json
2022-01-06T15:58:20.1924796Z 15:58:20.192  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/1/Issues.json
2022-01-06T15:58:20.2091972Z 15:58:20.208  Using longest common projects root path as project base directory: '/home/azure-agent/agent/_work/18/s'.
2022-01-06T15:58:20.2167456Z 15:58:20.216  15:58:20.216  WARNING: File '/tmp/.NETFramework,Version=v4.6.SqlClrAttributes.cs' is not located under the root directory '/home/azure-agent/agent/_work/18/s' and will not be analyzed.
2022-01-06T15:58:20.2168885Z 15:58:20.216  File was referenced by the following projects: '/home/azure-agent/agent/_work/18/s/AlignDatabase/AlignDatabase.sqlproj'.
2022-01-06T15:58:20.2234435Z 15:58:20.223  Dumping content of sonar-project.properties
2022-01-06T15:58:20.2235940Z ------------------------------------------------------------------------

And here’s with the new project:

2022-01-06T15:54:00.8484426Z ==============================================================================
2022-01-06T15:54:01.0058744Z [command]/home/azure-agent/agent/_work/_tool/dotnet/dotnet /home/azure-agent/agent/_work/_tasks/SonarCloudPrepare_14d9cde6-c1da-4d55-aa01-2965cd301255/1.23.1/dotnet-sonar-scanner-msbuild/SonarScanner.MSBuild.dll end
2022-01-06T15:54:01.0865693Z SonarScanner for MSBuild 5.4
2022-01-06T15:54:01.0869409Z Using the .NET Core version of the Scanner for MSBuild
2022-01-06T15:54:01.1561778Z Post-processing started.
2022-01-06T15:54:01.2326848Z 15:54:01.232  15:54:01.197  Uninstalling target: /home/azure-agent/agent/_work/18/.sonarqube/bin/targets/SonarQube.Integration.targets
2022-01-06T15:54:01.2327991Z 15:54:01.232  15:54:01.232  sonar.verbose=true was specified - setting the log verbosity to 'Debug'
2022-01-06T15:54:01.2331056Z 15:54:01.232  Loading the SonarCloud analysis config from /home/azure-agent/agent/_work/18/.sonarqube/conf/SonarQubeAnalysisConfig.xml
2022-01-06T15:54:01.2332065Z 15:54:01.233  Not running under TeamBuild
2022-01-06T15:54:01.2335307Z 15:54:01.233  Analysis base directory: /home/azure-agent/agent/_work/18/.sonarqube
2022-01-06T15:54:01.2335794Z Build directory: 
2022-01-06T15:54:01.2336443Z Bin directory: /home/azure-agent/agent/_work/18/.sonarqube/bin
2022-01-06T15:54:01.2337197Z Config directory: /home/azure-agent/agent/_work/18/.sonarqube/conf
2022-01-06T15:54:01.2337932Z Output directory: /home/azure-agent/agent/_work/18/.sonarqube/out
2022-01-06T15:54:01.2338698Z Config file: /home/azure-agent/agent/_work/18/.sonarqube/conf/SonarQubeAnalysisConfig.xml
2022-01-06T15:54:01.2356193Z 15:54:01.235  Generating SonarCloud project properties file to /home/azure-agent/agent/_work/18/.sonarqube/out/sonar-project.properties
2022-01-06T15:54:01.2648015Z 15:54:01.264  Setting analysis property: sonar.visualstudio.enable=false
2022-01-06T15:54:01.2728014Z 15:54:01.272  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/12/Issues.json
2022-01-06T15:54:01.2732119Z 15:54:01.273  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/2/Issues.json
2022-01-06T15:54:01.2758240Z 15:54:01.275  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/10/Issues.json
2022-01-06T15:54:01.2761735Z 15:54:01.275  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/3/Issues.json
2022-01-06T15:54:01.2766571Z 15:54:01.276  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/5/Issues.json
2022-01-06T15:54:01.2771520Z 15:54:01.276  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/13/Issues.json
2022-01-06T15:54:01.2775103Z 15:54:01.277  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/16/Issues.json
2022-01-06T15:54:01.2778560Z 15:54:01.277  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/7/Issues.json
2022-01-06T15:54:01.2780956Z 15:54:01.277  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/15/Issues.json
2022-01-06T15:54:01.2803665Z 15:54:01.28  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/6/Issues.json
2022-01-06T15:54:01.2807140Z 15:54:01.28  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/4/Issues.json
2022-01-06T15:54:01.2810537Z 15:54:01.28  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/0/Issues.json
2022-01-06T15:54:01.2819575Z 15:54:01.281  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/8/Issues.json
2022-01-06T15:54:01.2823207Z 15:54:01.282  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/1/Issues.json
2022-01-06T15:54:01.2826768Z 15:54:01.282  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/9/Issues.json
2022-01-06T15:54:01.2830179Z 15:54:01.282  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/14/Issues.json
2022-01-06T15:54:01.2834782Z 15:54:01.282  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: /home/azure-agent/agent/_work/18/.sonarqube/out/11/Issues.json
2022-01-06T15:54:01.3017745Z 15:54:01.301  Using longest common projects root path as project base directory: '/'.
2022-01-06T15:54:01.3179193Z 15:54:01.317  Dumping content of sonar-project.properties
2022-01-06T15:54:01.3180033Z ------------------------------------------------------------------------

I’m happy to send you the yaml and the full logs via DM. One other thing that might helpful is just to know what the actual directory names are. Maybe there’s something weird with the names? Here is the original structure:

/syz.api.services.sln
/syz.api.admin/
/syz.api.web/
/syz.api.unittest/

and here are the two directories added that lead to the problem (if they are referenced by the build–just adding them to the solution doesn’t cause an issue)

/align.function.feedback/
/align.function.feedback.test/

You’ll notice we did change our naming convention for these projects, though not sure why that would matter.