Review of configuration in Administration section (exclusions, languages extension setup)
Problem statement
My repo has folder structure as below.
root
|- api-prj (here C# solution)
|- app-prj (typescript SPA page)
Code gets detected only in “api-prj” folder (on properly setup long-living branch), also not all code gets identified in Lines of code column in “Code” section. Most of API controllers code and domain project login is not reported, tough Code smells are reported for those folders and there are numbers in the column.
1.1. I managed to enforce Sonar to report all Lines of code on short-living branch by moving all code to different folder, but no “Code smelle” where reported later. Next push zeroed columns.
1.2. After a couple of tries I got to a point where Sonar reports that Your project contains only TEST-code for language C# and no MAIN-code for any language, so only TEST-code related results are imported.. I’ve tried to use “<SonarQubeTestProject>false</SonarQubeTestProject>” for code and “true” for tests, but it ended up in only test projects being checked and all code projects being skipped.
I still can’t understand how to enforce Sonar to check the typescript folder. I’ve tried many “basefolders” setups (in API, in APP, in root…) and folder includes (“…/app-prj1”), but it didn’t help.
Some background taks id to check: AYnBB4MVsqFLeNBsDQPi
Regarding “Analyzing languages other than C# and VB” the first paragraph says that
“For newer SDK-style projects (used by .NET Core, .NET 5, and later), the SonarScanner for .NET will analyze all file types supported by the available language plugins unless explicitly excluded.”
Solution is in .NET 6, so it means that I don’t need to add files to “.csproj”. It should be scanned in “app-prj”, but it’s not. As mentioned earlier in “Problem statement” point 2
Regarding C# code, we had an issue of using FluenAssertions.Extensions in one of the project. This was fixed. Now I see only unit tests project as tests, but no other code. In the background work task id “AYnZQvAJhw1AegxNQH0-” you can see that there are many more files detected, but only unit tests get reported in the final long-lived branch analysis. Even unit tests have “smells” reported, but no information about detected “Lines of code”.
Other finding from “Analyse” log: “Found 20 MSBuild C# projects: 4 TEST projects. 16 with no MAIN nor TEST files.”. Why “no MAIN”? (Background task id: “AYnZlgvaNSfPCQrHw0Cf”)
Sample of logs for project that was skipped but shouldn’t (9 files skipped):
INFO: Sensor C# Project Type Information [csharp]
INFO: Sensor C# Project Type Information [csharp] (done) | time=0ms
INFO: Sensor C# Analysis Log [csharp]
INFO: Roslyn version: 4.3.0.0
INFO: Language version: CSharp10
INFO: Concurrent execution: enabled
INFO: Sensor C# Analysis Log [csharp] (done) | time=0ms
INFO: Sensor C# Properties [csharp]
INFO: Sensor C# Properties [csharp] (done) | time=0ms
INFO: Sensor TextAndSecretsSensor [text]
INFO: Sensor TextAndSecretsSensor [text] (done) | time=20ms
INFO: Sensor VB NET Project Type Information [vbnet]
INFO: Sensor VB NET Project Type Information [vbnet] (done) | time=0ms
INFO: Sensor VB NET Analysis Log [vbnet]
INFO: Sensor VB NET Analysis Log [vbnet] (done) | time=0ms
INFO: Sensor VB NET Properties [vbnet]
INFO: Sensor VB NET Properties [vbnet] (done) | time=0ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: ‘sonar.coverage.jacoco.xmlReportPaths’ is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacocoit/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=0ms
INFO: Sensor CSS Rules [javascript]
INFO: No CSS, PHP, HTML or VueJS files are found in the project. CSS analysis is skipped.
INFO: Sensor CSS Rules [javascript] (done) | time=1ms
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend]
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=0ms
INFO: Sensor IaC Docker Sensor [iac]
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC Docker Sensor [iac] (done) | time=5ms
INFO: Sensor Serverless configuration file sensor [security]
INFO: 0 Serverless function entries were found in the project
INFO: 0 Serverless function handlers were kept as entrypoints
INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
INFO: Sensor AWS SAM template file sensor [security]
INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
INFO: Sensor AWS SAM Inline template file sensor [security]
INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=1ms
I did read your response! And my understanding is that you manually checked the conditions, instead of looking at the debug logging which indicates why a project gets categorized the way it does. Did you bump up the log level as described, and if so can you share those logs?
I’ve bumped it even to debug, but the file is a bit to large to share (over 700MB). The only mentions about .csproj are in the beginning of the log. No other places. verbose-run-header.txt (8.1 KB)
There are many warnings about files from projects “not detected” by Sonar that goes like:
2023-08-08T13:24:16.5709193Z 13:21:25.754 WARN: File ‘/agent/_work/20/s/(filePath)’ referenced by the protobuf ‘CopyPasteTokenInfo’ does not exist in the analysis context
and later on:
2023-08-08T13:24:16.5797536Z 13:21:25.807 DEBUG: Skipping issue S107, input file not found or excluded: /agent/_work/20/s/(filePath)
In whole file I can’t find any string like “Categorizing project”, “test project”
apart from:
2023-08-08T13:24:16.4775034Z 13:21:25.588 WARN: SonarScanner for .NET detected only TEST files and no MAIN files for C# in the current solution. Only TEST-code related results will be imported to your SonarQube/SonarCloud project. Many of our rules (e.g. vulnerabilities) are raised only on MAIN-code. Read more about how the SonarScanner for .NET detects test projects (…)
and
2023-08-08T13:24:16.5801931Z 13:21:25.808 INFO: Found 20 MSBuild C# projects: 4 TEST projects. 16 with no MAIN nor TEST files.
Some interesting thing is around project sensors:
2023-08-08T13:21:38.0815606Z 13:21:13.015 DEBUG: Sensors : C# Project Type Information → C# Analysis Log → C# Properties → TextAndSecretsSensor → VB.NET Project Type Information → VB.NET Analysis Log → VB.NET Properties → JaCoCo XML Report Importer → CSS Rules → ThymeLeaf template sensor → IaC Docker Sensor → Serverless configuration file sensor → AWS SAM template file sensor → AWS SAM Inline template file sensor
2023-08-08T13:21:38.0816085Z 13:21:13.015 INFO: Sensor C# Project Type Information [csharp]
2023-08-08T13:21:38.0816816Z 13:21:13.016 DEBUG: Adding file type information (has MAIN ‘false’, has TEST ‘false’) for project ‘[Project]’ (project key ‘[key]’, base dir ‘/agent/_work/20/s/(path)’). For debug info, see ProjectInfo.xml in ‘/agent/_work/20/.sonarqube/out/17’.
(…)
2023-08-08T13:21:38.0818471Z 13:21:13.016 INFO: Roslyn version: 4.3.0.0
2023-08-08T13:21:38.0818700Z 13:21:13.016 INFO: Language version: CSharp10
2023-08-08T13:21:38.0818933Z 13:21:13.016 INFO: Concurrent execution: enabled
2023-08-08T13:21:38.0819388Z 13:21:13.016 DEBUG: File ‘/agent/_work/20/s/(path)/obj/Release/net6.0/.NETCoreApp,Version=v6.0.AssemblyAttributes.cs’ was recognized as generated
2023-08-08T13:21:38.0819947Z 13:21:13.016 DEBUG: File ‘/agent/_work/20/s/(path)/obj/Release/net6.0/(prj).AssemblyInfo.cs’ was recognized as generated
2023-08-08T13:21:38.0820459Z 13:21:13.016 DEBUG: File ‘/agent/_work/20/s/(path)/obj/Release/net6.0/(prj).MvcApplicationPartsAssemblyInfo.cs’ was recognized as generated
2023-08-08T13:21:38.0820837Z 13:21:13.016 INFO: Sensor C# Analysis Log [csharp] (done) | time=0ms
2023-08-08T13:21:38.0821081Z 13:21:13.016 INFO: Sensor C# Properties [csharp]
It shows that the projects are being processed, but not reported?
I switched verbosity to “normal” on build and all code projects (15 projects) are correctly identified as “production code”, but none are visible on “Code” tab.
09:59:10 [DBG] SonarCategoriseProject:
09:59:10 [DBG] Sonar: ([Project].csproj) Categorizing project as test or product code…
09:59:11 [DBG] Sonar: ([Project].csproj) categorized as MAIN project (production code).
09:59:11 [DBG] SonarCreateProjectSpecificDirs:
09:59:11 [DBG] Creating directory “/agent/_work/14/.sonarqube/conf/15”.
09:59:11 [DBG] SonarWriteFilesToAnalyze:
09:59:11 [DBG] Sonar: ([Project].csproj) Number of files to analyse: 22. The list of files to be analyzed is in /agent/_work/14/.sonarqube/conf/15/FilesToAnalyze.txt.
Message “Your project contains only TEST-code for language C# and no MAIN-code for any language, so only TEST-code related results are imported.” still visible after analysis.
One of the basic projects in solution had a reference to a nuget package that was identified as a test project reference making whole solution discovered as “test only”.
I tried to include TS/CSS analysis in the same project. I tried to include TS sources in path that eventually broke up C# analysis (it’s better to have path “include” empty). That was a bad choice. I should use monorepo configuration and setup API and APP solutions in separate projects.