Sonar scanner analysis is taking more time JS

Hi Team,
sonar scan is taking more time for JS.Can you please help us on this?

INFO: Sensor JsSecuritySensor [security]
INFO: Reading type hierarchy from: /data/var/lib/jenkins/workspace/ui-component/future/ui-pos-future/.scannerwork/ucfg2/js
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /data/var/lib/jenkins/workspace/ui-component/future/ui-pos-future/.scannerwork/ucfg2/js
INFO: 11:46:51.624765 Building Runtime Type propagation graph
INFO: 11:46:52.094788 Running Tarjan on 89503 nodes
INFO: 11:46:52.224526 Tarjan found 89488 components
INFO: 11:46:52.412542 Variable type analysis: done
INFO: 11:46:52.416121 Building Runtime Type propagation graph
INFO: 11:46:52.950375 Running Tarjan on 89503 nodes
INFO: 11:46:53.073212 Tarjan found 89488 components
INFO: 11:46:53.260808 Variable type analysis: done
INFO: Analyzing 12444 ucfgs to detect vulnerabilities.
INFO: Taint analysis starting. Entrypoints: 317
INFO: Running symbolic analysis for ā€˜JSā€™
INFO: Taint analysis: done.
INFO: Sensor JsSecuritySensor [security] (done) | time=1293295ms

Thanks,
Revanth

Hi Revanth,

Can you characterize ā€œmore timeā€?

How long did it take before the ā€œmoreā€? And when did it change?

Ā 
Ann

Hi @ganncamp ,
This is the first time we are adopting sonar for JS and it is taking around 27mins.This is not recommendedā€¦ we want to reduce this time.Please let us know is there any way to do thatā€¦

INFO: Analysis total time: 27:50.339 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 27:54.321s
INFO: Final Memory: 798M/2118M
Thanks,
Revanth

Hi,

Can you provide some sizing on your project? Things like:

  • LoC
  • file count
  • LoC of largest file
  • [anything else significant]

Ā 
Thx,
Ann

Hi @ganncamp ,

attached screenshot will give you more info on thisā€¦

But as per my observation, sonar scan is taking around 25 to 30mins depends on component but it is not recommended resulting increasing our total build timeā€¦

INFO: Reading UCFGs from: /data/var/lib/jenkins/workspace/ui-component/master/ui-pos-master/.scannerwork/ucfg2/js
INFO: 08:27:41.928212 Building Runtime Type propagation graph
INFO: 08:27:42.442767 Running Tarjan on 89503 nodes
INFO: 08:27:42.605034 Tarjan found 89488 components
INFO: 08:27:42.770673 Variable type analysis: done
INFO: 08:27:42.773851 Building Runtime Type propagation graph
INFO: 08:27:43.407623 Running Tarjan on 89503 nodes
INFO: 08:27:43.50274 Tarjan found 89488 components
INFO: 08:27:43.750804 Variable type analysis: done
INFO: Analyzing 12444 ucfgs to detect vulnerabilities.
INFO: Taint analysis starting. Entrypoints: 317
INFO: Running symbolic analysis for ā€˜JSā€™
INFO: Taint analysis: done.
INFO: Sensor JsSecuritySensor [security] (done) | time=1315785ms
Thansk,
Revanth

Hi @ganncamp ,

do you have any update on this?

Thanks,
Revanth

Hi,

Thanks for the details. Iā€™ve flagged this for team attention.

Ā 
Ann

Hi @ganncamp ,
Did we get any update on this?
Thanks.
Revanth

Hi revant,

How much RAM are you using in your scanner environment?
The security analysis is known to use more RAM, can you increase it in your environment?

Best regards,
Ilia

Hi @ilia ,

we are using 32GB RAM.I think this is sufficeā€¦ Please check attached screenshot for your reference.we have 13GB available and also 6GB buffer/cache available.

Thanks,
Revanth

1 Like

Hi @ilia ,

still this issue persist.Please let me know do you want me to check some other parameters or need to add any to decrease scanner timeā€¦

Thanks,
Revanth

Hi Revanth,

Could you please post the logs of the scanner now that you have increased the RAM?

Best Regards,
Ilia

Hi @ilia ,

i was trying to say, our environment has already 32GB RAMā€¦but still sonar scanner is taking more time to completeā€¦

jenkins build logs:

INFO: Sensor JsSecuritySensor [security]
INFO: Reading type hierarchy from: /data/var/lib/jenkins/workspace/ui-component/future/ui-pos-future/.scannerwork/ucfg2/js
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /data/var/lib/jenkins/workspace/ui-component/future/ui-pos-future/.scannerwork/ucfg2/js
INFO: 11:46:51.624765 Building Runtime Type propagation graph
INFO: 11:46:52.094788 Running Tarjan on 89503 nodes
INFO: 11:46:52.224526 Tarjan found 89488 components
INFO: 11:46:52.412542 Variable type analysis: done
INFO: 11:46:52.416121 Building Runtime Type propagation graph
INFO: 11:46:52.950375 Running Tarjan on 89503 nodes
INFO: 11:46:53.073212 Tarjan found 89488 components
INFO: 11:46:53.260808 Variable type analysis: done
INFO: Analyzing 12444 ucfgs to detect vulnerabilities.
INFO: Taint analysis starting. Entrypoints: 317
INFO: Running symbolic analysis for ā€˜JSā€™
INFO: Taint analysis: done.
INFO: Sensor JsSecuritySensor [security] (done) | time=1293295ms

sonar scanner logs:

SonarCloud plugins:

  • IaC Code Quality and Security 1.9.2.2279 (iac)
  • PL/SQL Code Quality and Security 3.8.0.4948 (plsql)
  • Scala Code Quality and Security 1.11.0.3905 (sonarscala)
  • C# Code Quality and Security 8.51.0.59060 (csharp)
  • Vulnerability Analysis 9.9.0-M1.18978 (security)
  • Java Code Quality and Security 7.16.0.30901 (java)
  • HTML Code Quality and Security 3.7.1.3306 (web)
  • Flex Code Quality and Security 2.8.0.3166 (flex)
  • XML Code Quality and Security 2.6.1.3686 (xml)
  • Text file Code Quality and Security 1.2.0.510 (text)
  • VB.NET Code Quality and Security 8.51.0.59060 (vbnet)
  • Swift Code Quality and Security 4.8.0.5759 (swift)
  • CFamily Code Quality and Security 6.41.0.60884 (cpp)
  • Python Code Quality and Security 3.23.0.10732 (python)
  • Dataflow Bug Detection Rules for Python 1.10.0.3046 (dbdpythonfrontend)
  • Dataflow Bug Detection 1.10.0.3046 (dbd)
  • Go Code Quality and Security 1.11.0.3905 (go)
  • JaCoCo 1.2.0.1505 (jacoco)
  • Kotlin Code Quality and Security 2.12.0.1956 (kotlin)
  • Dataflow Bug Detection Rules for Java 1.10.0.3046 (dbdjavafrontend)
  • T-SQL Code Quality and Security 1.7.0.5449 (tsql)
  • Apex Code Quality and Security 1.11.0.3905 (sonarapex)
  • JavaScript/TypeScript/CSS Code Quality and Security 9.12.1.20358 (javascript)
  • Ruby Code Quality and Security 1.11.0.3905 (ruby)
  • Vulnerability Rules for C# 9.9.0-M1.18978 (securitycsharpfrontend)
  • Vulnerability Rules for Java 9.9.0-M1.18978 (securityjavafrontend)
  • License for SonarLint 8.0.0.36792 (license)
  • Vulnerability Rules for JS 9.9.0-M1.18978 (securityjsfrontend)
  • COBOL Code Quality 5.2.0.5949 (cobol)
  • Vulnerability Rules for Python 9.9.0-M1.18978 (securitypythonfrontend)
  • PHP Code Quality and Security 3.27.0.9339 (php)
  • ABAP Code Quality and Security 3.11.0.4030 (abap)
  • Configuration detection fot Code Quality and Security 1.2.0.267 (config)
  • Vulnerability Rules for PHP 9.9.0-M1.18978 (securityphpfrontend)
    Global server settings:
  • delete_old_projects_deployment_date=1671634414000
  • delete_old_projects_excluded_project_kees=brave_brave-core,simgrid_simgrid,apache_struts,microsoft_vscode-python,mediawiki-core,jhipster-sample-application,JMeter,typo3,org.xwiki.platform:xwiki-platform,apache_ofbiz-framework,org.nuxeo:nuxeo-ecm,monica,sonarlint-visualstudio
  • email.from=noreply@sonarcloud.io
  • email.fromName=SonarCloud
  • email.prefix=[SonarCloud]
  • sonar.auth.bitbucket.enabled=true
  • sonar.auth.microsoft.enabled=true
  • sonar.core.id=1BD809FA-AWHW8ct9-T_TB3XqouNu
  • sonar.core.serverBaseURL=https://sonarcloud.io
  • sonar.core.startTime=2023-01-23T16:12:09+0100
  • sonar.dbcleaner.weeksBeforeDeletingAllSnapshots=260
  • sonar.dbcleaner.weeksBeforeKeepingOnlyOneSnapshotByMonth=4
  • sonar.dbcleaner.weeksBeforeKeepingOnlyOneSnapshotByWeek=1
  • sonar.global.exclusions=**/build-wrapper-dump.json
  • sonar.lf.enableGravatar=true
  • sonar.lf.logoWidthPx=105
  • sonar.maintenance_mode.link=SonarCloud Status - SonarCloud Scheduled Maintenance
  • sonar.maintenance_mode.message=Results of analyses performed prior to 6:15am CET may not be available yet and will be progressively provided throughout coming hours. Results of analysis performed after 8:00am CET are available.
  • sonar.maintenance_mode.start_date=2022-03-26T23:00:00.000+01:00
  • sonar.organizations.anyoneCanCreate=true
  • sonar.organizations.createPersonalOrg=true
  • sonar.plsql.file.suffixes=sql,tab,pkb
  • sonar.sensor.cache.baseUrl=https://ea6ne4j2sb.execute-api.eu-central-1.amazonaws.com/current
  • sonar.sensor.cache.enableForNewProjects=true
  • sonar.sensor.cache.loc.threshold=0
  • sonar.tsql.file.suffixes=.tsql
    Project server settings:
  • sonar.sensor.cache.project.enable=true
    Project scanner properties:
  • sonar.exclusions=/node_modules/
  • sonar.host.url=https://sonarcloud.io
  • sonar.login=******
  • sonar.nodejs.executable=/usr/bin/node
  • sonar.organization=*********
  • sonar.projectBaseDir=/data/var/lib/jenkins/workspace/ui-component/master/ui-pos-master
  • sonar.projectKey=ui_pos
  • sonar.projectName=ui-pos
  • sonar.scanner.app=ScannerCLI
  • sonar.scanner.appVersion=4.7.0.2747
  • sonar.sourceEncoding=UTF-8
  • sonar.sources=src
  • sonar.tests.inclusions=**/*.spec.ts
  • sonar.typescript.lcov.reportPaths=coverage/lcov.info
  • sonar.working.directory=/data/var/lib/jenkins/workspace/ui-component/master/ui-pos-master/.scannerwork

Thanks,
Revanth

Hi Revanth,

Can you try passing this environment variable when running the scanner?

export SONAR_SCANNER_OPTS="-Xmx20000m"

This increases the memory available to the analyzer. Can you try with different values like:

  • 10000
  • 20000
  • 25000

Best regards,
Ilia

Hi @ilia ,
i have set this environment variable (export SONAR_SCANNER_OPTS=ā€œ-Xmx35000mā€) to our environment but still it i taking more time to complete.I found that sensor JsSecuritySensor is taking more time.Please note that we are using Typescript and javascript language for this analysis.Also i observed we have around 317 .ucfg files in /data/var/lib/jenkins/workspace/ui-component/master/ui-pos-master/.scannerwork/ucfg2/js folder where as its not there in our source codeā€¦those were getting generated during sonar scan time

INFO: Reading UCFGs from: /data/var/lib/jenkins/workspace/ui-component/master/ui-pos-master/.scannerwork/ucfg2/js

INFO: 06:12:23.841933 Building Runtime Type propagation graph

INFO: 06:12:24.482438 Running Tarjan on 89718 nodes

INFO: 06:12:24.626604 Tarjan found 89703 components

INFO: 06:12:24.821712 Variable type analysis: done

INFO: 06:12:24.825115 Building Runtime Type propagation graph

INFO: 06:12:25.573286 Running Tarjan on 89718 nodes

INFO: 06:12:25.672136 Tarjan found 89703 components

INFO: 06:12:25.891891 Variable type analysis: done

INFO: Analyzing 12469 ucfgs to detect vulnerabilities.

INFO: Taint analysis starting. Entrypoints: 317

INFO: Running symbolic analysis for ā€˜JSā€™

INFO: Taint analysis: done.

INFO: Sensor JsSecuritySensor [security] (done) | time=1549818ms

Thanks,
Revanth

Hey @revant,

Iā€™ll jump in here.

Thank you for all the information you have provided so far. As @ilia mentioned, in-depth security analysis is something that requires more resources. However, ~25 minutes for ~130k lines of code is not something we aim for and as such would be interested to investigate this.

To help us investigate, would you be able to share an archive of the content of this folder with us? If thatā€™s possible, I would open a private discussion thread with you so that you donā€™t have to share them publicly.

In the meantime, I invite you to have a look at the documentation page ā€œNarrowing the focus with an analysis scopeā€ and verify that your project is configured accordingly. For example, if test files are correctly marked as such, not all rules will be run on them which should already make the analysis faster.

Hi @Karim_El_Ouerghemmi ,

Thank you for your response.Please give us sometime.I will check with management and get back to youā€¦

Thanks,
Revanth

1 Like

Hi @Karim_El_Ouerghemmi ,

we will share with you archive of ucfg filesā€¦please open a private discussion thread with us ā€¦

Thanks,
Revanth