Bitbucket yml pipeline PHP project scans JS sourcefiles for 40 min

SonarCloud
, ,
1

The problem:
The yml pipeline for my Laravel PHP project runs smoothly until we hit the
INFO: Sensor JavaScript analysis [javascript]
This starts scanning source files that are not present in my repo, but were apparently retrieved from Bitbucket and are add to Docker?
It starts node.js, which I did not asked the pipeline to do, and it starts scanning Atlassian source files:

INFO: Sensor JavaScript analysis [javascript]
DEBUG: Deploying bundle
DEBUG: Deploying eslint-bridge into /opt/atlassian/pipelines/agent/build/.scannerwork/.sonartmp/eslint-bridge-bundle
DEBUG: Deploying bundle (done) | time=2069ms
DEBUG: Deploying custom rules bundle jar:file:/root/.sonar/cache/fc422c4b061327b347a25963ee1affc1/sonar-securityjsfrontend-plugin.jar!/js-vulnerabilities-rules-1.0.0.tgz to /opt/atlassian/pipelines/agent/build/.scannerwork/.sonartmp/eslint-bridge-bundle/package/custom-rules5597528658977234149
DEBUG: Starting server
DEBUG: Using default Node.js executable: 'node'.
DEBUG: Checking Node.js version
DEBUG: Launching command node -v
DEBUG: Using Node.js v16.13.2.
DEBUG: Starting Node.js process to start eslint-bridge server at port 46151
DEBUG: Launching command node /opt/atlassian/pipelines/agent/build/.scannerwork/.sonartmp/eslint-bridge-bundle/package/bin/server 46151 127.0.0.1 /opt/atlassian/pipelines/agent/build/.scannerwork true false /opt/atlassian/pipelines/agent/build/.scannerwork/.sonartmp/eslint-bridge-bundle/package/custom-rules5597528658977234149/package
DEBUG: starting eslint-bridge server at port 46151
DEBUG: eslint-bridge server is running at port 46151
DEBUG: Starting server (done) | time=3460ms
DEBUG: Using generated tsconfig.json file /opt/atlassian/pipelines/agent/build/.scannerwork/.sonartmp/6273489293182686458.tmp
DEBUG: Analysis of unchanged files will not be skipped (current analysis requires all files to be analyzed)
INFO: 18 source files to be analyzed
DEBUG: Analyzing file: file:///opt/atlassian/pipelines/agent/build/public/assets/vendors/base/bootstrap-daterangepicker.js
DEBUG: Analyzing file "/opt/atlassian/pipelines/agent/build/public/assets/vendors/base/bootstrap-daterangepicker.js" with linterId "default"
INFO: 1/18 file analyzed, current file: /opt/atlassian/pipelines/agent/build/public/assets/vendors/base/bootstrap-daterangepicker.js
INFO: 13/18 files analyzed, current file: /opt/atlassian/pipelines/agent/build/public/assets/demo/default/base/tinymce/plugins/emoticons/js/emojis.js

This scan process of these source files takes at least 40 minutes!
The pipeline either breaks based on memory or a Docker timeout (after 120 min).

There are some simple front-end .js files in my codebase, so I need to keep scanning .js. But it does not fail on my own three .js files, it fails because it takes a long time to scan these weird 18 external source files.
The big question is: how do I turn this off? How do I skip this step? And why does it keep scanning these external files?
I already added * * /opt/ * * (spaces needed for formatting here) to my exclusion value, but that did not help. Of course.
This one stands out to me:
DEBUG: Analysis of unchanged files will not be skipped (current analysis requires all files to be analyzed)
So I guess somewhere there is an option to get rid of this?

My stats:

  • ALM used: Bitbucket Cloud
  • CI system used: Bitbucket Cloud
  • Languages of the repository: PHP, JS, CSS.

My yml is the basic from the onboard wizard for PHP projects.
My image: image: atlassian/default-image:2

And the steps I take:

  steps:
    - step: &build-test-sonarcloud
        size: 2x
        name: Sonar scan
        caches:
          - sonar
        script:
          - pipe: sonarsource/sonarcloud-scan:1.4.0
            variables:
              EXTRA_ARGS: '-Dsonar.organization=[REMOVED] -Dsonar.projectKey=[REMOVED] -Dsonar.sources=. -Dsonar.host.url=https://sonarcloud.io'
              DEBUG: "true"
    - step: &check-quality-gate-sonarcloud
        name: Check the Quality Gate on SonarCloud
        script:
          - pipe: sonarsource/sonarcloud-quality-gate:0.1.6
            variables:
              DEBUG: "true"

The rest of the file is default. The file sonar-project.properties is also default, straight from the onboarding wizard.
Can you let me know what I did wrong? How can I fix this issue?

2

Hi,

Welcome to the community!

The ** pattern excludes 0-n directories. You’ll need to add a file specifier (/* or /*.js), for a pattern of **/opt/**/*.js.

 
HTH,
Ann

3

Hi Ann,

Sadly this makes no change.
It still scans the SOURCE files of the Bitbucket docker setup.

17:46:47.355 INFO: 13/18 files analyzed, current file: /opt/atlassian/pipelines/agent/build/public/assets/demo/default/base/tinymce/plugins/emoticons/js/emojis.js

These are not my files. These are the files installed by the Docker build.

It made no difference when I add your suggestion via the Sonarcloud config (Source File Exclusions):

17:46:14.155 INFO: Project configuration:
17:46:14.156 INFO:   Excluded sources: **/build-wrapper-dump.json, public/hexon/**, public/pim/**, public/images/**, public/fonts/**, bitbucket-pipelines.yml, public/mix-manifest.json, **/opt/**/*.js, **/*.map, **/*.svg, **/*.yml, **/*.yaml

This happens during scanning (in - step: &build-test-sonarcloud).
This seems to be something different.
Why would Sonarcloud even scan files ourside my project that do not belong to the project?

Regards, Bas

4

Hi Bas,

Can you share the full analysis log?

The analysis / scanner log is what’s output from the analysis command. Hopefully, the log you provide - redacted as necessary - will include that command as well.

This guide will help you find them.

 
Ann

5

Hi Ann,

The pipeline (debug):
pipelineLog-{88ba3333-a63e-4db4-a7a0-d5dbe9c633e0}.txt (398.2 KB)

The sonar-project.properties:
sonar-project properties.txt (617 Bytes)

The bitbucket-pipelines.yml:
bitbucket-pipelines yml.txt (1.4 KB)

Hope to hear from you,
Bastiaan

6

Hi Bastiaan,

I believe this is the problem:

18:34:34.964 INFO: Base dir: /opt/atlassian/pipelines/agent/build

Analysis is executed from a directory inside /opt, so there’s no match with your **/opt/*.js pattern.

I’m not sure why you’re seeing the full path here:

DEBUG: Analyzing file: file:///opt/atlassian/pipelines/agent/build/public/assets/vendors/base/bootstrap-daterangepicker.js
DEBUG: Analyzing file "/opt/atlassian/pipelines/agent/build/public/assets/vendors/base/bootstrap-daterangepicker.js" with linterId "default"

But I believe the part of the path that’s relevant is this:
public/assets/vendors/base/bootstrap-daterangepicker.js

So try something like this pattern instead:
**/vendors/**/*.js

 
HTH,
Ann