After updating to version 1.1.0 oif sonarCloud my test folder is now included in the analysis

ralfonsoreyes · June 19, 2023, 3:19pm

I am using circleci orb to execute the scan action, after migration to the version 1.1.0, I was on version 1.03 before, the scan begin to analyse the test folder.

An extract of my project file:

sonar.language=js
sonar.sources=src
sonar.inclusions=src/**/*.ts
sonar.exclusions=src/**/configurationParametersMap.ts
sonar.tests=test
sonar.javascript.lcov.reportPaths=coverage/lcov.info

Now even rolling back to 1.0.3 doesn’t work.

Any ideas :frowning ?

ganncamp · June 20, 2023, 4:44pm

Hi,

Welcome to the community!

It looks like the latest version is 1.1.1. But what is the error you’re seeing? Can you share your analysis logs?

The analysis / scanner log is what’s output from the analysis command. Hopefully, the log you provide - redacted as necessary - will include that command as well.

This guide will help you find them.

Ann

ralfonsoreyes · June 20, 2023, 4:53pm

Hi,

Thanks for your reactivity!!

Please find here an extract of the logs:

INFO: Load project repositories
INFO: Load project repositories (done) | time=713ms
INFO: SCM collecting changed files in the branch
INFO: SCM collecting changed files in the branch (done) | time=258ms
INFO: Indexing files...
INFO: Project configuration:
INFO:   Included sources: ./src/**/*.ts
INFO:   Excluded sources: **/build-wrapper-dump.json, ./src/**/configurationParametersMap.ts
INFO: 43 files indexed
INFO: 64 files ignored because of inclusion/exclusion patterns
INFO: 0 files ignored because of scm ignore settings
INFO: Quality profile for js: Sonar way
INFO: Quality profile for json: Sonar way
INFO: Quality profile for ts: Sonar way
INFO: ------------- Run sensors on module configuration-parsers
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=480ms
INFO: Sensor cache enabled
INFO: Load sensor cache
INFO: Load sensor cache (99 KB) | time=1257ms
INFO: Sensor IaC CloudFormation Sensor [iac]
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC CloudFormation Sensor [iac] (done) | time=24ms
INFO: Sensor IaC Kubernetes Sensor [iac]
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC Kubernetes Sensor [iac] (done) | time=5ms
INFO: Sensor C# Project Type Information [csharp]
INFO: Sensor C# Project Type Information [csharp] (done) | time=1ms
INFO: Sensor C# Analysis Log [csharp]
INFO: Sensor C# Analysis Log [csharp] (done) | time=30ms
INFO: Sensor C# Properties [csharp]
INFO: Sensor C# Properties [csharp] (done) | time=0ms
INFO: Sensor TextAndSecretsSensor [text]
INFO: Sensor TextAndSecretsSensor is restricted to changed files only
INFO: Sensor TextAndSecretsSensor [text] (done) | time=62ms
INFO: Sensor VB.NET Project Type Information [vbnet]
INFO: Sensor VB.NET Project Type Information [vbnet] (done) | time=0ms
INFO: Sensor VB.NET Analysis Log [vbnet]
INFO: Sensor VB.NET Analysis Log [vbnet] (done) | time=14ms
INFO: Sensor VB.NET Properties [vbnet]
INFO: Sensor VB.NET Properties [vbnet] (done) | time=0ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=2ms
INFO: Sensor JavaScript/TypeScript analysis [javascript]
INFO: 39 source files to be analyzed
INFO: 1/39 file analyzed, current file: /home/circleci/project/test/filters/GL320M.test.ts
INFO: 2/39 files analyzed, current file: /home/circleci/project/test/filters/GL501M.test.ts
INFO: 4/39 files analyzed, current file: /home/circleci/project/test/filters/GL300N.test.ts
INFO: 5/39 files analyzed, current file: /home/circleci/project/test/parsers/GV500MA.test.ts
INFO: 7/39 files analyzed, current file: /home/circleci/project/test/repository/deviceConfiguration.test.ts
INFO: 8/39 files analyzed, current file: /home/circleci/project/test/api/getIniConfigurationByImeiOrDeviceId.test.ts
INFO: 10/39 files analyzed, current file: /home/circleci/project/test/parsers/GL310M.test.ts
INFO: 12/39 files analyzed, current file: /home/circleci/project/test/parsers/GL300.test.ts
INFO: 13/39 files analyzed, current file: /home/circleci/project/test/parsers/genericParser.test.ts
INFO: 16/39 files analyzed, current file: /home/circleci/project/test/filters/Validation.test.ts
INFO: 18/39 files analyzed, current file: /home/circleci/project/test/parsers/GV50M.test.ts
INFO: 19/39 files analyzed, current file: /home/circleci/project/test/filters/GV500MA.test.ts
INFO: 21/39 files analyzed, current file: /home/circleci/project/test/parsers/GL502MG.test.ts
INFO: 24/39 files analyzed, current file: /home/circleci/project/test/parsers/GL300M.test.ts
INFO: 25/39 files analyzed, current file: /home/circleci/project/test/filters/GL520M.test.ts
INFO: 27/39 files analyzed, current file: /home/circleci/project/test/filters/GV57MG.test.ts
INFO: 28/39 files analyzed, current file: /home/circleci/project/test/parsers/GL320M.test.ts
INFO: 30/39 files analyzed, current file: /home/circleci/project/test/parsers/GL300N.test.ts
INFO: 31/39 files analyzed, current file: /home/circleci/project/test/filters/GB130.test.ts
INFO: 33/39 files analyzed, current file: /home/circleci/project/test/filters/GL521M.test.ts
INFO: 34/39 files analyzed, current file: /home/circleci/project/test/filters/GL300.test.ts
INFO: 37/39 files analyzed, current file: /home/circleci/project/test/utils/clearDynamoDbTable.ts
INFO: 38/39 files analyzed, current file: /home/circleci/project/test/api/getConfigByImeiOrDeviceId.test.ts
INFO: 39/39 source files have been analyzed
INFO: Hit the cache for 0 out of 39
INFO: Miss the cache for 39 out of 39: FILE_CHANGED [39/39]
INFO: Sensor JavaScript/TypeScript analysis [javascript] (done) | time=238444ms

Regards
Ruben

ganncamp · June 20, 2023, 5:14pm

Hi Ruben,

Your parameters in the first post include sonar.tests, but I don’t see that reflected in your analysis log. Are you sure those are your current analysis parameters/settings?

I think it’s worth double-checking because I see another discrepancy:

Your OP

Versus your analysis log:

That leading ./ on the inclusion is a minor discrepancy, but analysis woudln’t have added it in on its own. It’s another indication that maybe you’re not running with the parameters you think you are.

Note that you can use sonar.scanner.dumpToFile=[path to file] to see the final parameter list analysis is running with.

Ann

ralfonsoreyes · June 21, 2023, 7:08am

Hi Ann,

I tested different configurations, that’s why the difference.

Please find attached the verbose output of CircleCi with the original post configuration.

Regards
Ruben
outputCircleCi.zip (192.0 KB)

ganncamp · June 21, 2023, 2:39pm

HI,

Here’s what I see when I open your log file:

Ann

ralfonsoreyes · June 21, 2023, 6:43pm

Hi

Please find the same file in UTF8

Ruben
outputCircleCiUTF8.zip (192.0 KB)

ralfonsoreyes · June 22, 2023, 8:52am

Hi,

In the logs i found that maybe this new problem, last week everything worked OK, is that something change an for some reason now I am seeing this: 08:45:15.118 DEBUG: Created program for /home/circleci/project/tsconfig.json with 1610 files. that not normal because my tsconfig file only has “include”: [“./src/**/*.ts”],
“exclude”:

Regards
Ruben

ganncamp · June 22, 2023, 1:21pm

Hi Ruben,

Thanks for re-sending the log file.

I’m seeing lines like this:

06:54:06.503 DEBUG: 'test/filters/GB130.test.ts' indexed as test with language 'ts'

So yes, the files under test are included, but they’re not included as source files, but as test files, according to the log. And that’s expected.

I guess having test files included at all was a change? And that’s what made you question what’s going on?

Ann

ralfonsoreyes · June 22, 2023, 4:29pm

Hi,

The real problem is the time, normally its take 2 mins max, now its taking more than 5 mins some times 7 mins without any code change, and no more files. Maybe the " /home/circleci/project/tsconfig.json with 1610 files" is what’s taking more time than before?

I also tested with this configuration:

sonar.language=js
sonar.sources=src/
sonar.inclusions=src/**/*.ts
sonar.exclusions=test/**,src/**/configurationParametersMap.ts
sonar.tests=test/
sonar.test.exclusions=tests/**,src/**/configurationParametersMap.ts
sonar.javascript.lcov.reportPaths=coverage/lcov.info
# workaround for avoid memory problem
sonar.javascript.node.maxspace=4096
sonar.verbose=true
``
Without any change with the time.

Regards

Ruben

ganncamp · June 22, 2023, 6:35pm

Hi,

Okay, you’re probably interested in this thread

And as a side note

sonar.langauge=js = this isn’t doing anything
** in an exclusion/inclusion pattern means 0-n directories, so sonar.exclusions=test/** isn’t doing what you think it is. Try sonar.exclusions=test/**/*.ts (&etc) instead
except that your specification of sonar.tests=test/ naturally already excludes those files from being considered source files.

Ann

ralfonsoreyes · June 22, 2023, 8:35pm

Hi,

Thanks for the link, that’s the solution.

I could come back to my simple original solution:

sonar.sources=./src
sonar.inclusions=./src/**/*.ts
sonar.exclusions=./src/**/configurationParametersMap.ts
sonar.tests=./test

Only adding the

sonar.typescript.tsconfigPath=tsconfig.sonar.json

Kind regards
Ruben