Analyzing only the changed files in sonarqube

Hi,

How can we analyze only the changed files in repo as currently whenever changes is pushed in repo it will analyze the whole project again. If there is any way which can help me achieving the same then refer the documentation else this can be considered as suggestion for new feature as this process consume lot of time unnecessarily.
Thanks in advance!

Hi @vicky,

You are posting a question inside of a Suggest new features category, that’s a bit surprising. To answer your question, we currently do not support incremental analysis.

Cheers,
Amaury

hi,

well we are using somekind of hacky workarround, but this involves some magic handled by the build tool, and the passing of some information to that build tool. like pull request key or something.

What we are doing is triggering our feature build with an pull request number, and getting the changeset information via api from the pullrequest. We parse it, and add just the changes sources to the inclusion property of sonar. in this case, we just analyse the files we changed.

There are some downsides for some plugins though, eg. spotbugs/findbugs ignores the inclusion and will still analyse the whole source code.

we also found this way not ideal, because it requires configurations on 3 different systems. but in the end, it speed up our pull request code analysis, so for us it was worth the effort.

But that is just an suggestion how you could solve that issue, althought not ideal.

Cheers
simon

You can get the list of changed files from the github api and then plug it into sonar.inclusions .
Simple.

In context to jenkins,

  1. create a sonar.properties file (using shell)
  2. make a github api call and fetch the list of files changed with their api.
  3. insert the above fetched value inside the sonar.properties file (sonar.inclusions=) (open file in write mode)
  4. and pass this sonar file to the build configuration (maven, ant etc)

In this way , there would be new sonar inclusions for each PR in jenkins.

Hi,

Can you please help me with how to dynamically add PR files to sonar.inclusions when doing an ANT build. I do not use Jenkins or any CI tool for my project. I tried searching for this online, but was unable to find any help.

Thanks

We do not advise doing this kind of things (this is why we don’t try to make it easy to do): Each file cannot be seen as an independent entity. Changes in one source file can impact the way the code in another file is interpreted. Changes ripple through the full code base.

If you just analyse the subset of modified files, you will miss those changes, which will lead to errors in the analysis. Depending on the analyzed language, you may also have other issues, because a partial analysis cannot work correctly with all rules (for instance, you will probably have lots of false positives for rules detecting unused methods).

We recently introduced incremental analysis for C, C++ and Objective-C languages. This incremental analysis correctly tracks all dependencies and makes sure project-level rules work correctly. We plan to extend this to other languages.

2 Likes

I get the sonar stance on this but there is definitely some value in doing an incremental scan for possible security issues upfront on a reduced scope on a feature branch. Then knowing that a full scan of the whole project can happen later.

I really wish Sonar would reconsider not making this so hard or at least finding ways to support parrallel scanning or multithreading in more languages. For even descent sized Java projects scans can easily take 10/15 minutes. Folks really want to include a scaning (even a partial one) into their CI/CD pipelines and that’s just impossible with a 10m scan time. And before you say it I have done the whole debug your sonar scan and try to speed it up so not looking for help there (It just takes 10m). However 10m is just too long for a pipeline.

I think most folks are willing to take the risk of a false positive for the sake of catching a hard security early on.