Very slow javascript/typescipt analysis for project

SonarQube Cloud
,
1

Around the end of June / early July, analysis time went from ~3 minutes to 25 minutes (577 vue/ts files). I found some of the other topics…

…and followed the workarounds, however they haven’t made any difference to the runtime.

We’re using azure devops and our prepare analysis task is as follows…

- task: SonarCloudPrepare@1
      displayName: Prepare analysis on SonarCloud
      inputs:
        SonarCloud: SonarCloud
        organization: ${orgName}
        projectKey: ${projectKey}
        projectName: ${projectName}
        projectVersion: '$(Build.BuildNumber)'
        extraProperties: |
                    sonar.cs.vscoveragexml.reportsPaths=$(Agent.TempDirectory)\TestResults\*.coverage            sonar.coverage.exclusions=**/*Tests.cs,**/*Test.cs,**/Startup.cs,**/Program.cs,**/Routes.cs,**/ServiceCollectionExtensions.cs,**/*Policies.cs
                    sonar.cs.vstest.reportsPaths=$(Agent.TempDirectory)\TestResults\*.trx
                    sonar.typescript.tsconfigPath = tsconfig.sonar.json

It does look like we’ve never managed to get caching working properly for PR analysis, but that’s a 2nd issue. I’d like to know if there’s anything else we can look at to try and improve the analysis times.

2

Hi,

Can you post your analysis logs, please?

The analysis / scanner log is what’s output from the analysis command. Hopefully, the log you provide - redacted as necessary - will include that command as well.

This guide will help you find them.

 
Ann

3

Apologies, the debug output for the analysis task is exceptionally large. Would it be possible to send you the output directly? Mainly as I’m not sure which bits are needed or not and don’t want to sanitise useful info.

5

Hi,

I’m going to flag this for the language experts. They can set up a private transfer or guide you on which portions to post.

 
Ann

6

Ah ok, thank you. Hopefully there’s a solution!

7

I’ve done some more investigation… It appears that vue files are particularly slow to scan, however it’s considerably worse when they’re in multiple directories (standard web project). The interesting thing here is that it doesn’t matter for js or ts files, only vue.

If we remove all the vue files from the normal project structure and place them in a singular directory, the scanning is ~5x faster (we went from 25 minutes down to 5 for the same number of files).

So I’m currently trying to work around the issue by altering the file structure post build, but before scanning places. This appears to come with its own set of issues though, the scanner thinks they’re not part of the project and now ignores them. Any help would be appreciated.

1 Like
8

Hi,

Thanks for this great investigation!

This topic is still in queue for the language experts, and I’m sure it will be helpful when they get there.

You shouldn’t have to do this. :frowning:
Hopefully we’ll get this figured out soon.

 
Ann

9

Bit of an update, I think I’ve gone as far as I can with attempts at circumventing the issue. I was able to move the vue files to a single directory between the build and analysis stages. I then added them to the correct “FilesToAnalyze.txt” for the project.

This resulted in a large reduction in scan time for the same number of files (it looks like the scanner crawls the directories and checks against the list rather than going by the list in the first instance?).

However, there doesn’t appear to be a way to hack it into the report that gets uploaded, or I couldn’t spot anything. I’m guessing this is something to do with the path not matching the structure in the branch.

1 Like
11

Hi,

Just wondering if we could get any help on this one at all? The flat structure on the .vue files solved the issue in terms of scanning time, but we can’t alter the project structure to achieve this. When I circumvented it and had it scan the files when moving them post msbuild, and adding them to the FilesToAnalyze, unfortunately they were missed off the web report.

Just wondering if we need to raise an issue, or if there’s another route we can take to get around the problem.

Thanks
James

1 Like
12

hello @JamesF, sorry for the late reply. Could you please post part of the logs related to Vue analysis? It should start with

INFO: Sensor JavaScript/TypeScript analysis [javascript]

and ends with

INFO: Sensor JavaScript/TypeScript analysis [javascript] (done)
13

Hi Tibor,

I’ve got the logs (with debug on), but there’s 5000 lines, is there an appropriate way to post this or do you want a sampling, there are a number of typescript files mixed in. I also have logs from the flattened structure also, do you want both? Or just the slow one?

Thanks
James

16

we’ve created a ticket here Improve performance on Vue.js projects · Issue #4352 · SonarSource/SonarJS · GitHub to track the issue. I linked this thread to it

1 Like