Javascript Code Coverage not working when using MSBUILD Scaner

I have a project which includes C# code and Javascript. I want to see code coverage for both the C# tests and the Javascript tests, however, I’m not managing to get the javascript test coverage included.

I created a simple project (in the above mentioned github repository) which has a C# console application and some javascript tests.

If I run the azure pipeline in MSBUILD mode (https://github.com/rubenmamo/SonarCoverage/blob/master/azure-pipelines.yml) I do not get the Javascript code, tests and coverage visible in sonar cloud (refer to RubenMamoTestCoverageMSBUILD)

If I run the azure pipeline in CLI mode (https://github.com/rubenmamo/SonarCoverage/blob/master/azure-pipelines-cli.yml) I get the javascript code, tests and coverge visible in sonar cloud but then I don’t have the C# code visible (refer to RubenMamoTestCoverageCLI)

Has anyone gotten such a project to work?

Hi @rubenmamo and welcome to the community !

C# code base can only be analysed with the Scanner for MSBuild, otherwise, you won’t get any issues on SonarCloud.

That being said, for the moment (but please be sure that we are working on it !) you’ll have to have different SonarCloud project and multiple analysis to ensure that all of your code of any kind of language.

By the way (but if you create multiple projects you won’t need it), multiple coverage properties and path should work with the Scanner for MSBuild.

Let us know.

Thanks
Mickaël

We have a similar setup but using git in Azure rather than github.

Our project successfully analyses C#, JS and typescript.

Our main .sln includes various csproj files, one of which is the main website so references all of our js files as well (I think this might key to getting the analyser to work?).

We run js tests from the command line (npm run test essentially) and then output a test result and lcov file:
We add some custom reporters to format the output correctly and calculate code coverage.

ng test project --reporters=junit,progress,coverage-istanbul --nowatch --source-map=false --code-coverage

These files are then setup in the sonar project under the ‘Tests and Coverage’ setting once you’ve selecting the language.

So the build pipeline looks like this

  • Start Analysis
  • Run MS Build
  • Run JS / TS Tests
  • Run Unit Tests
  • Run Code Analysis ad submit to SQ

Hope that helps, happy to answer any questions if you have any.

Matt

@matt_sharpe, thanks for the reply.

The links I shared are for a sample project not the actual one I had issues. (I cannot share that project unfortunately)

  • Do you have code coverage on both the c# and javascript tests?
  • Do you use karma for javascript testing?
  • What format do you use for the code coverage file? I’ve tried both lcov and cobertura and added the files in sonar.javascript.lcov.reportPaths, however only the C# code coverage is being loaded.

I’ll try another sample project with a web app instead of a console app to see whether I can resolve it

Hi Ruben,

Yeah we’ve got code coverage on C#, JS and TS.

We use MSTest v2 so just toggle to code coverage setting in the test runner, this outputs a trx file with test results and coverage. So we usesonar.cs.vstest.reportsPaths and set it to **/*.trx.

For JS we’re using the Angular CLI which wraps karma to call the jasmine tests.

sonar.javascript.lcov.reportPaths is the property that points to our generated lcov

I can share the config file for karma as it’s just settings: https://gist.github.com/mattsharpe/e2e7b950de275d4e744aaff0cac3d447

The key here is the karma runner uses istanbul to output coverage data but we have to tell it which format to use so we’ve got a few different output types:

 coverageIstanbulReporter: {
      dir: require('path').join(__dirname, './test/coverage'),
      reports: ['html', 'lcovonly', 'text-summary'],
      fixWebpackSourcePaths: true
    },

We did have a bit of rinse and repeat getting all the paths correctly lined up from build agent -> karma -> Sonar.

The wildcards help in the sonar properties **\*.lcov, might not be the most efficient but works!

Thanks, I’ll test it out.

Just a quick update. I updated the repository to have an MVC project and a Unit Tests project instead of the console app.

I still can’t get the MSBUILD version to load the coverage for the JS files even though I have the path to the lcov.info file correct (I know it’s correct because I use the same location when running with the CLI and it loads it correctly)

@matt_sharpe are your javascript tests in the web project or in a separate unit tests project?

They’re in the same project as the website so they share node modules etc. and are stripped out for release.

I don’t think it should matter as long as the lcov path is set right.

It might be worth running in debug and going through the log file with a fine tooth comb to verify sonar is checking the right place for your lcov information

I did run it with debug. The proerty is listed in the log, but there is nothing else mentioning the coverage.

sonar.organization=rubenmamo
sonar.cs.vscoveragexml.reportsPaths=D:\\a\\_temp\\TestResults\\4c2ce19c-0528-49ce-88d4-d6f035dcef7d\\VssAdministrator_fv-az776 2019-11-28 16_07_10.coveragexml
sonar.host.url=https://sonarcloud.io/
sonar.projectKey=RubenMamoTestCoverageMSBUILD
sonar.projectName=RubenMamoTestCoverageMSBUILD
sonar.projectVersion=1.0
sonar.scanner.metadataFilePath=D:\\a\\_temp\\sonar\\20191128.10\\686a6c17-40f4-1bb3-d757-a7feda7ef530\\report-task.txt
sonar.javascript.lcov.reportPaths=SampleWebApp.Tests/reports/coverage/lcov.info
sonar.visualstudio.enable=false

sonar.modules=DBA59751-D172-4748-AB60-11B630DB8BDE,FEED4CCB-B049-4ED9-A812-F45621D644C6

The full log can be found here https://dev.azure.com/rubenmamo/SonarCoverage/_build/results?buildId=42 if anyone is interested in having a look.

I modified the build to dump the whole working directory to a file and I have found the file in the location

Hello @rubenmamo,

Note that when importing coverage for a file, this step will happen when the file itself is being analyzed. In your example, when your javascript file (SampleWebApp/wwwroot/js/site.js) is being analyzed, the working directory is the SampleWebApp directory as the analyzer is currently analyzing this SampleWebApp project.

Looking at the full log you’ve linked, in the logs for the SonarCloudAnalyze job, we can see the following:

16:08:48.277 INFO: Sensor SonarJS Coverage [javascript]
16:08:48.277 INFO: 1/1 source files have been analyzed
16:08:48.277 WARN: No coverage information will be saved because LCOV file cannot be found.
16:08:48.277 WARN: Provided LCOV file path: SampleWebApp.Tests/reports/coverage/lcov.info. Seek file with path: D:\a\1\s\SampleWebApp\SampleWebApp.Tests\reports\coverage\lcov.info
16:08:48.277 WARN: No coverage information will be saved because all LCOV files cannot be found.

So the report coverage that is attempted to be loaded, is relative to the SampleWebApp working directory, which is why it cannot find the lcov.info report.
Using sonar.javascript.lcov.reportPaths=../SampleWebApp.Tests/reports/coverage/lcov.info should solve your issue.

Let me know if this fixes your problem.

Best,
-Chris

@czurn, thanks for the suggestion. I will test it out and will give an update

Hello @rubenmamo,

Were you able to resolve the issue on your side following the previous suggestion ?

Thanks,
-Chris