T-SQL Not getting scanned for L and XL

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
  • Developer Edition
  • Version 8.3.1 (build 34397)
  • LGPL v3
  • what are you trying to achieve
    -Scan .sql code in our codebase as T-SQL code using the Developer edition language support for T-SQL

  • what have you tried so far to achieve this

  • Upgrade to Developer edition and purchase of relevant license as per lines of code in use.
  • sonar.tsql.file.suffixes = .sql from UI
  • Scanning is done using Azure DevOps Task for Sonar Scanning. Prepare, Run and Publish all the 3 tasks are there.

Problem : Folders containing T-SQL .sql code and the Database solution does not get scanned by the sonar scanner or show up in the code section of the project.

Folder Structure:
Aurigo.Masterworks.Terminologies.WISDOT
Aurigo.Masterworks.WISDOT
Aurigo.Masterworks.WISDOT.Database
Aurigo.Masterworks.WISDOT.ResourceManagement

In Sonar only the following code shows up
Aurigo.Masterworks.Terminologies.WISDOT/Properties
Aurigo.Masterworks.WISDOT

SQL script within these folders do show up and are analysed I presume. But Database folder is not scanned.

Tried the following to forcefully include the Database folder:

  1. Added sonar.sources.
    sonar.sources=Aurigo.Masterworks.Terminologies.WISDOT,Aurigo.Masterworks.WISDOT ,Aurigo.Masterworks.WISDOT.Database,Aurigo.Masterworks.WISDOT.ResourceManagement

There is an error with indexing

03:22:05.184 INFO: Indexing files of module 'wisdot'

03:22:05.184 INFO: Base dir: ***

03:22:05.184 DEBUG: Source paths: Aurigo.Masterworks.WISDOT, Aurigo.Masterworks.WISDOT.Database, Aurigo.Masterworks.Terminologies.WISDOT, Aurigo.Masterworks.WISDOT.ResourceManagement
03:22:05.197 INFO: ------------------------------------------------------------------------
03:22:05.197 INFO: EXECUTION FAILURE
03:22:05.197 INFO: ------------------------------------------------------------------------

#[error]File Aurigo.Masterworks.WISDOT/AddStandardTable.aspx can't be indexed twice. Please check that inclusion/exclusion patterns produce disjoint sets for main and test files

Keeping Aurigo.Masterworks.WISDOT in sources is giving the same error as above.
Mentioning only the Database folder name seems to forefully include DB Scripts. But why is that necessary for certain project and not necessary for the other. Please help us understand

  1. Followed this SO post :https://stackoverflow.com/questions/32227360/multi-module-project-analysis-with-sonarqube to setup mulitmodule
    sonar.modules=Aurigo.Masterworks.WISDOT,Aurigo.Masterworks.WISDOT.Database

    Aurigo.Masterworks.WISDOT.sonar.projectName=Aurigo.Masterworks.WISDOT
    Aurigo.Masterworks.WISDOT.sonar.sources=Aurigo.Masterworks.WISDOT
    Aurigo.Masterworks.WISDOT.Database.sonar.projectName=Aurigo.Masterworks.WISDOT.Database
    Aurigo.Masterworks.WISDOT.Database.sonar.sources=Aurigo.Masterworks.WISDOT.Database

Scanning failed :
ERROR: Invalid value of sonar.sources for wisdot:Aurigo.Masterworks.WISDOT

Also, How can I find the number of LOC getting added as part of the scan? I am getting the error for exceeding the 2M LOC limit but wish to know how much more LOC I am pushing in as part of the new scan.

Hi Soham,

Can you please share a full log of the analysis in debug mode (passing the -X option to the scanner) without any special “sources” property (so that you do not get the failure of double indexing)?

I want to check first if you have no exclusions interfering or if you are not ignoring files through a .gitignore file. The full log will help clarifying a lot.

For your second question, LOC are computed by summing up the LOC of each project analyzed. The LOC count for a project is the LOC count of the project’s largest branch, except for lines that only include comments. You can see how many LOCs are added or removed in a project’s analysis history. In SonarQube, go to the project you are interested on, click on “Activity”, above the graph in the dropdown menu choose “Custom” and then add the metric “Lines of Code”. You will see how LOCs evolve for each analysis on the selected branch (you should select the largest branch, usually the main branch!).

Hello Daniel,

Thanks for the quick response in reviewing the issue. Apologies for the delay in getting back.

I have the logs collected from multiple projects. I am attaching it as a zip.

To explain the content of the zip, here it is –

There are 3 folders,

KCMetro, WisDOT and Product

In WisDOT there are two subfolders capturing logs with and without specifying the sonar.sources explicitly to point to the database folder.

Scenario:

Sonar-Scan-Logs.zip (794 KB)

There are 3 folders,

KCMetro, WisDOT and Product

In WisDOT there are two subfolders capturing logs with and without specifying the sonar.sources explicitly to point to the database folder.

Scenario:

  1. KCMetro works perfectly. T-SQL code is getting scanned by default. No need for any extra parameters. In the logs all the database sql scripts get listed.

  2. WisDOT:

  3. If no extra parameter is present, only c#, html, xml, css get scanned. SQL scripts are excluded from scanning.

  4. If sonar.sources= is explicitly added [sonar.sources=Aurigo.Masterworks.WISDOT.Database], the sql scripts get listed and scanned via T-SQL plugin

  5. Product: - Similar behavior as WisDOT is observed. If Database folder is explicitly mentioned, sql scripts are getting scanned. Else only c#, css, xml, html are getting scanned.

Please let me know if you need any further info.Sonar-Scan-Logs.zip (793.7 KB)

Hi Daniel,

Can you please check the logs already shared ?
Can you please guide me how to pass -X option in Azure DevOps tasks.

Hi Soham,

From the logs, it seems like your “Database” folder is on your project but not properly added to the Solution (.sln) in Visual Studio, and therefore the scanner is not able to find it.

You do not need to (and should not) pass any “sources” nor “modules” option in sonar.properties. As this is a .NET solution, the MSBuild scanner triggered in the Azure DevOps build job will use the Solution file (.sln) to map your sources. Therefore I would recommend you to make sure your “Database” and any other folders are correctly added/linked in your solution through Visual Studio.

Hope it helps, regards,
Daniel

Hello Daniel,

Thanks for the response,

I have reached out to Developers to check your suggestion.

In the meantime, I was checking the sln files for working and non working projects myself, I could not find anything that is drastically different. (In text compare)

Attaching the two for your reference. Could you please point me out to anything specific I should be checking.
Aurigo.Masterworks.KingCounty.sln.txt (5.5 KB) Aurigo.Masterworks.WISDOT.sln.txt (2.8 KB)

I would recommend to compare your text there are many tools available such as Text Compare.

That will show you the difference and helps to figure out the issue.