SonarCloud Scan: Coverage/Test Problem

rafael-loggi · July 4, 2023, 12:54pm

Hi,

We were facing some issues with sonarcloud scan, a week ago. More details HERE

ALM used: GitHub
CI: CircleCI
Scanner command used: - sonarcloud/scan
Language: JS

After that we noticed that some coverage information startsed to failed.

Information Log:
WARN: File '/opt/loggi/workspace/targets/wms/node_modules/@loggi/ui’ is ignored. It is a symbolic link targeting a file that does not exist.

INFO: Sensor Python Sensor [python] (done) | time=672ms
INFO: Sensor Cobertura Sensor for Python coverage [python]
INFO: Sensor Cobertura Sensor for Python coverage [python] (done) | time=6694ms
INFO: Sensor PythonXUnitSensor [python]
INFO: Sensor PythonXUnitSensor [python] (done) | time=6399ms

ERROR: Failure during analysis, Node.js command to start eslint-bridge was: node

INFO: Sensor JavaScript/TypeScript analysis [javascript] (done) | time=123356ms

WARN: Could not resolve 1 file paths in [/opt/loggi/workspace/targets/beyond/coverage/lcov.info]
WARN: First unresolved path: src/service-worker.js (Run in DEBUG mode to get full list of unresolved paths)
WARN: Found 332 inconsistencies in coverage report. Re-run analyse in debug mode to see details.

rafael-loggi · July 5, 2023, 5:58pm

Anyone?

rafael-loggi · July 10, 2023, 8:46pm

Up…
Someone can help us?

ganncamp · July 11, 2023, 1:15pm

Hi,

Are you saying that after you

set sonar.typescript.tsconfigPaths to a single tsconfig file

then your coverage import, which previously worked, started failing?

Ann

rafael-loggi · July 11, 2023, 3:06pm

Hi, Ann!

Changes from our side, YES.
This was a wokaround to issue with the scan time.

Regards,

victor.diez · July 12, 2023, 2:54pm

Hi @rafael-loggi,

the changes for the analysis should not affect coverage.

Did the coverage/lcov.info change in the meantime? Do you have the full debug logs to see the details?

Thanks,
Victor

raphaelcarvalho · July 12, 2023, 6:33pm

Hi Victor! I work together with Rafael

We have a script that generates our LCOV but it hasn’t been modified for a while. About the script, our LCOV files are only created when tests for the APP or LIB are run, but the tests in the PR only run for the APPs and LIBs with some diff, so LCOV files are only created in those cases.

Some fragments from a last scan that went through the same issue, FYI these dots ‘…’ are actually being used to omit sensitive/non-relevant information, but we have multiple paths here, I hope it helps anyway:

INFO: Sensor JavaScript/TypeScript Coverage [javascript]
INFO: Analysing .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info]
WARN: Could not resolve 424 file paths in [.../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, /opt/loggi.../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info, .../lcov.info]
WARN: First unresolved path: ... (Run in DEBUG mode to get full list of unresolved paths)
WARN: Found 783 inconsistencies in coverage report. Re-run analyse in debug mode to see details.
INFO: Sensor JavaScript/TypeScript Coverage [javascript] (done) | time=378ms

Another log example:

INFO: No CSS, PHP, HTML or VueJS files are found in the project. CSS analysis is skipped.
INFO: Sensor CSS Rules [javascript] (done) | time=12ms
INFO: Sensor JavaScript/TypeScript Coverage [javascript]
INFO: No LCOV files were found using .../lcov.info
INFO: Analysing [/.../lcov.info]
WARN: Could not resolve 1 file paths in [/.../lcov.info]
WARN: First unresolved path: src/service-worker.js (Run in DEBUG mode to get full list of unresolved paths)
WARN: Found 332 inconsistencies in coverage report. Re-run analyse in debug mode to see details.
INFO: Sensor JavaScript/TypeScript Coverage [javascript] (done) | time=11788ms

rafael-loggi · July 13, 2023, 11:59am

Hi, @victor.diez

We saw the new release #10.3.2 (Bugfix release to fix performance regression on projects with many tsconfig.json files.) for SonarJS on Git and try to rollback the workaround, removing the sonar.typescript.tsfconfigPaths from propierties, and the tsconfig.sonar.json file.

We try to run tests and we did not see any improvements. The SCAN keeps “stuck” on the analysis.

INFO: Sensor JavaScript/TypeScript analysis [javascript] (done) | time=2899161ms
INFO: Sensor JavaScript inside YAML analysis [javascript]

And got some errors like this:
ERROR: eslint-bridge Node.js process is unresponsive. This is most likely caused by process running out of memory. Consider setting sonar.javascript.node.maxspace to higher value (e.g. 4096).
ERROR: Failed to get response while analyzing file:…

INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 50:40.672s
INFO: Final Memory: 77M/202M
INFO: ------------------------------------------------------------------------

victor.diez · July 13, 2023, 1:07pm

Hi @rafael-loggi,

the bugfix release has not yet been deployed in sonarcloud. You can expect it to be done probably tomorrow.

you can check current analyzer versions here.

Victor

victor.diez · July 13, 2023, 1:52pm

Hi @raphaelcarvalho,

can you please confirm the relative paths inside the lcov.info files have not changed format? Probably ‘baseUrl’ was different in your prod tsconfig.json? ‘baseUrl’ should not have any effect on lcov.info paths, but that’s the only effect I can imagine right now. I understand you only used tsconfig.sonar.json for the analysis right?

There should not be any relation between the workaround put in place during the bugged release and the coverage report, but we could maybe wait until sonarcloud deploys the bugfix release. If it’s fixed, then for sure there is a relation between the import of coverage reports and the sonar.typescript.tsfconfigPaths property.

Cheers,
Victor

rafael-loggi · July 13, 2023, 2:30pm

Hi @victor.diez!

We do not have changes on this configs. Just the workaround.

on scripts/sonar-config-params.sh we have:

DEFAULT_PATH="/coverage/lcov.info"

SONAR_LCOV_PARAM="sonar.javascript.lcov.reportPaths=${coverage_paths}"
echo -e "\n${SONAR_LCOV_PARAM}" >> sonar-project.properties

on sonar-project.properties we have:
sonar.typescript.tsconfigPath=tsconfig.sonar.json

And tsconfig.sonar.json:

{
  "compilerOptions": {
    "allowJs": true,
    "noImplicitAny": true
  }
}

We are gonna try to apply the fix on 10.3.2 to test the coverage removing the sonar.typescript.tsconfigPath configuration. This is a MonoRepo with many application.

victor.diez · July 17, 2023, 4:35pm

Hi guys,

the new release is already on SonarCloud. Did you notice any improvement?

Cheers,
Victor

rafael-loggi · July 17, 2023, 4:40pm

Hi Victor!

We saw through the link you sent to us, we’ve already applied the fix.

We are testing before send it to prod. I believe that by tomorrow we´ll have update.

We will let u know ASAP.
Thanks.

rafael-loggi · July 19, 2023, 4:33pm

Hi @victor.diez!
We are still testing with this version 10.3.2. But found some errors.

We run the scan with:

sonar.log.level=DEBUG
sonar.verbose=true

And got this:
15:04:44.056 INFO: Sensor JavaScript/TypeScript Coverage [javascript]
15:04:44.056 DEBUG: Property sonar.javascript.lcov.reportPaths is used.
15:04:44.056 DEBUG: Using ‘components/coverage/lcov.info’ to resolve LCOV files
15:04:55.771 INFO: No LCOV files were found using components/coverage/lcov.info
15:04:55.772 DEBUG: Using ‘targets/beyond/coverage/lcov.info’ to resolve LCOV files
15:04:55.772 INFO: Analysing [/…/…/…/…/…/coverage/lcov.info]
15:04:55.807 DEBUG: Problem during processing LCOV report: can’t save DA data for line 111 of coverage report file (java.lang.IllegalArgumentException: Line with number 71 doesn’t belong to file App.js).
15:04:55.807 DEBUG: Problem during processing LCOV report: can’t save DA data for line 112 of coverage report file (java.lang.IllegalArgumentException: Line with number 76 doesn’t belong to file App.js).
15:04:55.807 DEBUG: Problem during processing LCOV report: can’t save DA data for line 113 of coverage report file (java.lang.IllegalArgumentException: Line with number 78 doesn’t belong to file App.js).
15:04:55.807 DEBUG: Problem during processing LCOV report: can’t save DA data for line 114 of coverage report file (java.lang.IllegalArgumentException: Line with number 80 doesn’t belong to file App.js).
15:04:55.807 DEBUG: Problem during processing LCOV report: can’t save DA data for line 115 of coverage report file (java.lang.IllegalArgumentException: Line with number 83 doesn’t belong to file App.js).
…
15:04:55.838 DEBUG: Problem during processing LCOV report: can’t save BRDA data for line 15130 of coverage report file (java.lang.IllegalArgumentException: Line with number 45 doesn’t belong to file …js).
…
15:04:55.838 DEBUG: Problem during processing LCOV report: can’t save DA data for line 15330 of coverage report file (java.lang.IllegalArgumentException: Line with number 99 doesn’t belong to file …js).
15:04:55.838 DEBUG: Problem during processing LCOV report: can’t save BRDA data for line 15348 of coverage report file (java.lang.IllegalArgumentException: Line with number 81 doesn’t belong to file …js).
15:04:55.838 DEBUG: Problem during processing LCOV report: can’t save BRDA data for line 15349 of coverage report file (java.lang.IllegalArgumentException: Line with number 81 doesn’t belong to file …js).
…
15:04:55.905 WARN: Could not resolve 1 file paths in [/opt/…/…/…/…/coverage/lcov.info]
15:04:55.906 DEBUG: Unresolved paths:
src/service-worker.js
15:04:55.906 WARN: Found 347 inconsistencies in coverage report. Re-run analyse in debug mode to see details.
15:04:55.906 INFO: Sensor JavaScript/TypeScript Coverage [javascript] (done) | time=11850ms

victor.diez · July 21, 2023, 9:15am

Hi @rafael-loggi,

thanks for the information. Then it seems those errors indeed were not related to the changes in tsconfig files, as we did rollback those in the last release.

The ‘Line number…’ errors appear when a lcov report refers to a line which above the referred sourcefile max line. For example:

15:04:55.807 DEBUG: Problem during processing LCOV report: can’t save DA data for line 111 of coverage report file (java.lang.IllegalArgumentException: Line with number 71 doesn’t belong to file App.js).

means that, according to the scanner, the file App.js has less than 71 files. Can you please double check that?

Can you check line 111 of the lcov report? there seems to me right now the inconsistencies may come from the lcov report.

Cheers,
Victor

rafael-loggi · July 21, 2023, 12:23pm

Hi @victor.diez!

I had noted that.
With full debug we can find where is the problem.

Thank you for the support.

victor.diez · July 21, 2023, 12:42pm

Excellent, thanks for confirming @rafael-loggi

Enjoy the weekend

rafael-loggi · July 28, 2023, 2:36pm

Hi @victor.diez!

Just to let you know that we solve the issue with Coverage.

We change the package.json to:
"test:coverage": "npm run craco -- test --watchAll=false --coverage --maxWorkers=4 --logHeapUsage",

And we changed some files and folders. Everything worked.
The scanner could not see some FilePath and recognize others FileNames.

Regards

victor.diez · July 28, 2023, 2:59pm

Thanks for the information @rafael-loggi,

hope this can help other users with a similar problem.