Potential bug introduced with v8.0.0 of dotnet-sonarscanner (can’t be indexed twice)

I have a well tested set of GitHub Actions integrating with SonarCloud.

Since yesterday, with surprising release of v8.0.0 of dotnet-sonarscanner, my project fails hard do to some can’t be index twice error.

Here is the error:

INFO: ------------------------------------------------------------------------

552INFO: EXECUTION FAILURE

553INFO: ------------------------------------------------------------------------

554INFO: Total time: 13.470s

555ERROR: Error during SonarScanner execution

556INFO: Final Memory: 12M/60M

557INFO: ------------------------------------------------------------------------

558ERROR: File src/Savvyio.Extensions.EFCore.Domain/EfCoreAggregateDataSource.cs can’t be indexed twice. Please check that inclusion/exclusion patterns produce disjoint sets for main and test files

559ERROR:

560The SonarScanner did not complete successfully

56123:51:19.428 Post-processing failed. Exit code: 1

562Error: Process completed with exit code 1.

I will try to revert back to 7.1.1, as this and previous versions worked fine.

My workflow: savvyio/.github/workflows/pipelines.yml at main · codebeltnet/savvyio · GitHub
My Sonar action: sonarcloud-scan/action.yml at main · codebeltnet/sonarcloud-scan · GitHub

In advance thanks for any help that leads to resolving this issue.

1 Like

Hey there,

Could you share the verbose logs?
This is done by specifying /d:sonar.verbose=true in the begin step.

Thanks in advance!

Hi @gregory.paidis,

I will enable my verbose logs for my next run - for now the important thing was to verify if this is a breaking change with v8.0.0 - and I can confirm that it is.

Feel free to clone my repo (GitHub - codebeltnet/savvyio: The Savvy I/O Framework provides a SOLID and clean .NET class library for writing DDD, CQRS and Event Sourcing applications.) and investigate yourself in the meantime - I am just happy that I was able to patch my dependency without running my workflow from scratch: GitHub - codebeltnet/dotnet-tool-install-sonarscanner: An opinionated GitHub Action for installing sonar scanner tool as part of your CI flow.

Something has for sure changed between v7.1.1 and v8.0.0.

When I have a new PR with new code I will try to enable verbose log as well as using v8.0.0.

Cheers.

Hey,

There indeed probably is something broken on our side, as v8.0.0 was a pretty big change (hence the major bump).
We have a few more cases like yours in community, and the more info the better to investigate it.
It’s awesome that you shared your repo, I will try to reproduce the behavior locally and get back to you :slight_smile:

1 Like

Indeed - and this was a good way for me to not rely on latest-and-greatest version with the default setting for dotnet tool install :slight_smile:

Happy “bug-hunting” - and thank you for providing the best tool for quality awareness in the FOSS community.

Hi @gimlichael,

I have tried to reproduce the problem both on Ubuntu and Windows in my local environment without any success.

In both cases, the analysis works without problems.

The logs in the verbose mode will help us a lot to understand the root cause and provide support for your workflow.

Best,
Costin

Hi @costin.zaharia @gregory.paidis,

Even I had the same issue after the new version got updated, when I reverted to the older version of 7.1.1 it started working without any problems, I am using an Ubuntu-based image to Build the pipeline in AWS CI (CodeBuild) Repository (CodeCommit).

My Concern is before 8.0 it was working fine from last two years without any issues, apart from this, there is no configuration change

2 Likes

Hi @gimlichael and @elumalaic160,

Could you please share with us the logs in the verbose mode.

You can activate verbose mode by specifying /d:sonar.verbose=true in the begin step.

Hi @costin.zaharia,

I have generated the verbose and see it has our library and class details, the organization is not comfortable to sharing it. can you please suggest any alternate way.

Note: We are using Team plan subscription for Sonarcloud

Hi @elumalaic160,

I understand the concerns. Please feel free to remove any confidential information from the file.

If it would help, I can open a private discussion, and you could attach the file to that private discussion. Would that work for you?

Thanks,
Costin

Thanks @costin.zaharia, Yes lets we will have private discussion.

Thanks
Elumalai C

For sure - it has just been high priority to have the ball rolling, why I am currently using last stable version prior to 8.0.0. I will make a PR and play around … and see if I can share the result with you. I hope to do it either later today or early next week.

2 Likes

Sadly, it still fails with the same error (even in 8.0.1). What should I do about the log? I don’t know how to copy the text, as it only copies what is in screen (even I marked all 7.000 lines).

I figured it out - here you go: https://productionresultssa8.blob.core.windows.net/actions-results/8586364f-2594-4d51-93e9-313def23a02f/workflow-job-run-10ee4ff2-0ff4-5322-a924-c293dc8cecf4/logs/job/job-logs.txt?rsct=text%2Fplain&se=2024-08-23T16%3A38%3A21Z&sig=p94aErwaIas6Nsey95ZmLKX2VjHX%2BQXwmENRJOeWZ7Q%3D&ske=2024-08-24T02%3A03%3A48Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2024-08-23T14%3A03%3A48Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2024-05-04&sp=r&spr=https&sr=b&st=2024-08-23T16%3A28%3A16Z&sv=2024-05-04

Just to be clear; I will rollback to last stable version again.
If you want to investigate beyond the logs, feel free to fork or clone my repo - its MIT.
204_🔬 Code Quality Analysis.log (4.1 MB)

1 Like

Hey there,

I finally figured out what happened here, by forking your repo and playing with the scanner version.
On v8.0.0, we introduced multi-language support, as well as changed a bit how the projectBaseDir is calculated.

Basically, the projectBaseDir in your project used to be savvyio/savvyio/src/ in v7.1.0 and now it is savvyio/savvyio/ in v8.0.0.

The problem here is that you are manually setting sonar.sources=src/.

  • In v7.1.0, this means that your sonar.sources are ignored, as the projectBaseDir, namely savvyio/savvyio/src/, does not contain a directory src/.

  • On v8.0.0, the projectBaseDir is savvyio/savvyio/, which indeed DOES contain src/.
    So in this version we are considering src/ as a viable directory and the end step enumerates the files. This comes in conflict with our own file detection, as we are detecting the files that were part of the compilation and some of them are inside src/.

Can I ask you if you could remove the “defaults” found here, and instead rely on our automatic detection?

We are thinking on how to proceed with sonar.sources and sonar.tests, as we do not properly support them right now.

Last but definitely not least, I want to thank you for making this thread and linking your OSS repo, as it was pivotal in finding the erroneous behavior and making our product better.

3 Likes

Thank you for getting back on this; I am slightly confused, though.

It looks like what you are communicating is not aligned with your documentation: Analysis parameters & SonarQube

All my projects are using /src for source code - and /test for unit-/functional tests.

Will this work if I decide to remove these two properties (sonar.sources/sonar.tests) and use v8+ going forward?

If memory serves me right, this was the only way I could be 100% sure, that SonarCloud would not mix test code with source code. Eventhough all test code has the sonar exclude and other attributes set, sometimes it did not honor this … and the only “peace-of-mind” way to do it, was to use said two properties.

Even this (although C++), suggest to use sonar.sources: Understanding the analysis

2 Likes

Hey,

You are absolutely right, the documentation is not aligned with this.
This is a bit of an edge case, as usually .NET analysis does not require sources and tests to be set, as they are automatically populated. For this reason, the behavior was flaky at best and the documentation is not quite right.

With the v8.0.0 release, this became even more apparent, as the projectBaseDir is now set to the working directory.

If you find the time, could you please do a test with removing the sources and tests and let me know if the behavior is different than expected?

I will for sure have it in mind - I have quite a number of FOSS projects, so my concern is it might work for one but not for the others (where this src/test has proven worked every time - except latest SonarCloud MS Build thingy), and it time of writing, I am slightly overwhelmed with tasks at hand.

That written - I will keep you posted when time allows for it.

Keep up the good work; SonarCloud has a positive impact to keep code clean!

2 Likes

Hey there!

We recently released v9.0.
In that version, we are ignoring user-supplied sonar.sources and sonar.tests.

Based on how the scanner for .NET works, there is no reason to use these parameters.

In fact, if you used them before v8.0, it was at best a no-op and at worst a double-index failure.
After v8.0, it resulted almost always to a double index error.
After v9.0, the bug was fixed and the program should no longer crash.

In any case, to simplify your pipeline I would suggest not including these parameters to your begin step.

Cheers, thanks for the report!
It was quite helpful :slight_smile: