Slow .tsx analysis

Hi, I have also a similar problem. The engine hangs on a very small index.ts file (29 lines). Disabling jasmin.internal doesn’t seem to help.

How can we fix it? It is not acceptable that our pipeline spends 80% of its time scanning 1 file. It is just too long and adds unnecessary costs.

This is out current Bitbucket pipeline:

image: public.ecr.aws/sam/build-nodejs22.x:1.161

clone:
  depth: full

definitions:
  caches:
    sonar: ~/.sonar/cache 
    pnpm: $BITBUCKET_CLONE_DIR/.pnpm-store
  services:
    docker-with-large-memory:
      memory: 5120
      type: docker     
  steps:
  - step: &build-test-sonarcloud
      name: Build, Test and Scan with SonarQube Cloud
      services: [docker-with-large-memory]
      size: 8x
      caches:
        - pnpm
        - sonar
      script:
        (...)       
        - pnpm install
        - pnpm run test:coverage
        - pnpm run build
        - pipe: sonarsource/sonarcloud-scan:4.0.0
          variables:
              EXTRA_ARGS: >
                -Dsonar.jasmin.internal.disabled=true -Dsonar.verbose=true
        - pipe: sonarsource/sonarcloud-quality-gate:0.1.6
      artifacts:
        - dist/**

And these are the output logs:

14:59:48.086 DEBUG Analyzing /opt/atlassian/pipelines/agent/build/apps/custom-ui/calendar-app/src/index.tsx (59/105)
15:07:26.110 DEBUG Analyzing /opt/atlassian/pipelines/agent/build/apps/custom-ui/calendar-app/src/pages/Administration.tsx (60/105)
15:07:26.111 INFO  Analysis progress:  57% (60/105 files)

Thanks for the help

Hi,

You’ve resurrected a topic that’s 1 year old. Per the FAQ, please don’t do that.

I’ve moved your post to a new thread. Can you please flesh out the details like your context for this? I.e. are you on SonarQube Cloud? SonarQube for IDE (flavor and version)? SonarQube self-managed (flavor and version)?

Can you provide a full analysis log?

The analysis / scanner log is what’s output from the analysis command. Hopefully, the log you provide - redacted as necessary - will include that command as well.

This guide will help you find them.

 
Ann

Hi @ganncamp ,

It is hard to redact a file with 55K lines. What information do you actually need from the logs?

This is the command tha BitBucket is running:
docker container run
--volume=/opt/atlassian/pipelines/agent/build:/opt/atlassian/pipelines/agent/build
--volume=/opt/atlassian/pipelines/bin/docker:/usr/local/bin/docker:ro
--volume=/opt/atlassian/pipelines/agent/ssh:/opt/atlassian/pipelines/agent/ssh:ro
--volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes
--volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/sonarsource/sonarcloud-scan:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/sonarsource/sonarcloud-scan
--workdir=$(pwd)
--label=org.bitbucket.pipelines.system=true
--env=BITBUCKET_STEP_TRIGGERER_UUID="$BITBUCKET_STEP_TRIGGERER_UUID"
--env=BITBUCKET_REPO_FULL_NAME="$BITBUCKET_REPO_FULL_NAME"
--env=BITBUCKET_GIT_HTTP_ORIGIN="$BITBUCKET_GIT_HTTP_ORIGIN"
--env=BITBUCKET_PROJECT_UUID="$BITBUCKET_PROJECT_UUID"
--env=BITBUCKET_PACKAGES_TOKEN="$BITBUCKET_PACKAGES_TOKEN"
--env=BITBUCKET_REPO_IS_PRIVATE="$BITBUCKET_REPO_IS_PRIVATE"
--env=BITBUCKET_PACKAGES_USERNAME="$BITBUCKET_PACKAGES_USERNAME"
--env=BITBUCKET_WORKSPACE="$BITBUCKET_WORKSPACE"
--env=BITBUCKET_SSH_KEY_FILE="$BITBUCKET_SSH_KEY_FILE"
--env=BITBUCKET_REPO_OWNER_UUID="$BITBUCKET_REPO_OWNER_UUID"
--env=BITBUCKET_BRANCH="$BITBUCKET_BRANCH"
--env=BITBUCKET_REPO_UUID="$BITBUCKET_REPO_UUID"
--env=BITBUCKET_PROJECT_KEY="$BITBUCKET_PROJECT_KEY"
--env=BITBUCKET_REPO_SLUG="$BITBUCKET_REPO_SLUG"
--env=CI="$CI"
--env=BITBUCKET_REPO_OWNER="$BITBUCKET_REPO_OWNER"
--env=BITBUCKET_STEP_RUN_NUMBER="$BITBUCKET_STEP_RUN_NUMBER"
--env=BITBUCKET_BUILD_NUMBER="$BITBUCKET_BUILD_NUMBER"
--env=BITBUCKET_TEST_METADATA_FILE_PATH="$BITBUCKET_TEST_METADATA_FILE_PATH"
--env=BITBUCKET_GIT_SSH_ORIGIN="$BITBUCKET_GIT_SSH_ORIGIN"
--env=BITBUCKET_PIPELINE_UUID="$BITBUCKET_PIPELINE_UUID"
--env=BITBUCKET_PIPELINES_VARIABLES_PATH="$BITBUCKET_PIPELINES_VARIABLES_PATH"
--env=BITBUCKET_COMMIT="$BITBUCKET_COMMIT"
--env=BITBUCKET_CLONE_DIR="$BITBUCKET_CLONE_DIR"
--env=BITBUCKET_STEP_UUID="$BITBUCKET_STEP_UUID"
--env=BITBUCKET_STEP_OIDC_TOKEN="$BITBUCKET_STEP_OIDC_TOKEN"
--env=BITBUCKET_DOCKER_HOST_INTERNAL="$BITBUCKET_DOCKER_HOST_INTERNAL"
--env=DOCKER_HOST="tcp://host.docker.internal:2375"
--env=BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes"
--env=BITBUCKET_PIPE_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/sonarsource/sonarcloud-scan"
--env=EXTRA_ARGS="-Dsonar.jasmin.internal.disabled=true -Dsonar.verbose=true -Dsonar.coverage.exclusions=src/frontend/index.jsx,src/frontend/teamsFieldView.jsx,apps//src/index.tsx,src/index.js,/.config.,/vite.config.*,packages/types/src/
"
--env=SONAR_TOKEN="${SONAR_TOKEN}"
--add-host="host.docker.internal:$BITBUCKET_DOCKER_HOST_INTERNAL"
sonarsource/sonarcloud-scan:4.0.0

Hi,

At a minimum, I need to see the full header describing your environment and then from when .tsx analysis starts.

 
Ann

Hi @ganncamp ,

Please find attached the logs

Best regards,

Bruno

pipelineLog-redacted.txt (582.1 KB)

Hi @bmarotta-ease, I took a look at the logs. The engine takes about 3 minutes to analyze ~25k NCLOC, which falls within our performance targets.

I believe the confusion comes from the following part:

The engine hangs on a very small index.ts file (29 lines). […]. It is not acceptable that our pipeline spends 80% of its time scanning 1 file.

Unfortunately, analysis time does not necessarily scale linearly with the number of lines of code. Looking at the file names, it seems to be a React application. It is expected that most of the analysis time is spent on index.ts because it is likely the place where everything fits together. This is where the engine simulates all the complex interactions between hooks and components, which can only be done once the engine has created an abstract model of your application.

I hope this helps!

Hi @Samuele_Buro ,

Thanks for your answer. I must disagree with 2 points.

First, you stated that the engine takes about 3 minutes, which is not correct. You can see in the logs:

10:50:42.960 DEBUG Scanner status reported
10:50:42.992 INFO Analysis total time: 5:09.880 s

So it takes 5 minutes and not 3. I’m not sure where you got the 3 minutes from. And according to your performance targets, 100k LOC should take <= 5 minutes, not 25k.

Second, the performance targets also states that the code change analysis should be proportional to the change set. That’s not what we observe. We see that all analysis (master branch, PR merge or feature branch) take around 5 minutes.

Is it something that we need to change in our configuration?

Another disturbing fact is that we are already using a docker machine with bigger memory than all the other pipelines and the analysis seems to be hitting 100% memory. Should we increase the memory even more? What is the suggested memory for analysing a React app?

Thanks in advance,

Hi @bmarotta-ease,

The 3-minute figure I mentioned comes specifically from the JsSecuritySensorV2 (the JS/TS security analysis), which drives the bulk of the analysis time.

However, you are right that the total time includes other components. Specifically, the standard JS sensor (for bug and code smell detection) takes another ~30 seconds. The remaining time is non-plugin overhead (file indexing, downloading quality profiles, loading rules, …).

Second, the performance targets also states that the code change analysis should be proportional to the change set.

As stated on that documentation page, while this is our target, it does not currently apply to all languages. JavaScript and TypeScript are explicitly listed there under “next steps”. Delivering this incremental analysis for JS/TS is a primary focus for our team this year.

What is the suggested memory for analysing a React app?

This is highly application-dependent, so we do not have a specific baseline suggestion. Since you are hitting 100% memory usage, you should try increasing the allocation to see if performance improves.

Anyway, if you are willing to share your source code (or even better, a small reproducer) we can take a closer look to see whether we can improve the engine’s memory usage for your setup. Let me know if you’d like me to open a private thread for you to share those files securely.