Task fails and falsly claims maximum LoC reached

[uchendu.myky (Uch)]

Hi

We have trouble running a task on SonarCloud through our GitLab CI. Apparantly, we are hitting the max LoC, but this seems wrong. Our plan includes 200k LoC, our two projects have a total of slightly more than 100k LoC.

As a temporary fix, we just upgraded our plan to 300k LoC. The upgrade will end after a month and we have set it back to the original limit of 200k LoC.

The task fails with the following error message:
“This analysis will make your organization ‘org’ to reach the maximum allowed lines limit (having 200531 lines). \nPlease contact the administrator of the organization to resolve this issue.”

What is the issue here?

Best,
Uchendu

[ganncamp (G Ann Campbell)]

Hi Uchendu,

Welcome to the community!

This error indicates that the most recent analysis found more code that apparently you were expecting. It’s unfortunate that there’s not much information to help you diagnose this. At a guess, you’ve accidentally included a new library or generated code in your analysis. You may get a few clues from your analysis logs as to what’s being wrongly included in analysis. If not, take a close look at sonar.sources and any inclusions / exclusions.

 
HTH,
Ann

[uchendu.myky (Uch)]

We use once /d:sonar.scanner.scanAll=false for a project where we have 41k LoC. This was also during the time of issue not larger.

For the second repo we actually have an exclusion rule like -Dsonar.exclusions=libs/frontend/org/api/**that is the same since a year. The files under this path were not moved.

If you like, I could share the specific sonar cloud task ID of a failing request.

What is interesting, the screenshots above are after increasing the max LoC. So, technically 200k should have been enough from the start, but it was not.

Let me know, if we can provide any other information.

[ganncamp (G Ann Campbell)]

Hi,

I’m not sure what to tell you. The error is clear.

Your initial message mentioned one repo. Now you’re talking about two. Are both repos failing analysis with the same message or just one? And if so, which one?

 
Thx,
Ann

[uchendu.myky (Uch)]

To be very honest, I have not a full and clear understanding of your billing setup.

Our situation: as an organization, we have 2 projects. Both failed with the exact same message as mentioned above. The billing is configured for the whole organization.

I share the believe, that the error is clear. However, I also believe that my messages are showing clearly, that we do not have more than 200k LoC in our organization. This is why I am even posting here.

Please let me know, what kind of information I should provide to you. Our billing will be reset to 200k LoC in a week. We fear, that the same error pops up, since we haven’t changed anything in our codebase to get rid of the error, but only increased the LoC to 300k. Since we valued the analysis performed, these are blocking quality gates in our pipeline and when failing, disrupt our teams workflow.

[ganncamp (G Ann Campbell)]

Hi,

@uchendu.myky

We use once /d:sonar.scanner.scanAll=false for a project where we have 41k LoC

Is this a .NET project? Have there been significant structural changes in it recently?

@uchendu.myky

For the second repo we actually have an exclusion rule like -Dsonar.exclusions=libs/frontend/org/api/**

Can you try again with -Dsonar.exclusions=libs/**/*?

And yes, please send me that ID.

 
Thx,
Ann

[uchendu.myky (Uch)]

Hi Ann,

Thanks for your reply. I have to apologize, the error only concerned the project with 41k LoC. This is indeed a .NET project. The failed tasks have IDs such as: AZqlMcsl7HeA0p2Ay6ha or AZqiQg9z0URUPU5yuDrk.

The associated code changes are relatively small. A few code additions only, no files moved nor dependencies introduced or updated.

I will keep your suggestion for the second repository in mind, in case that we have another issue anytime soon.

Best,
Uchendu

[ganncamp (G Ann Campbell)]

Hi Uchendu,

For the .NET project, can you provide your analysis log?

Share the Scanner for .NET verbose logs

Add /d:"sonar.verbose=true" to the Scanner BEGIN step. Examples:

dotnet sonarscanner begin /k:"MyProject" /d:"sonar.verbose=true"

SonarScanner.MSBuild.exe begin /k:"MyProject" /d:"sonar.verbose=true"

- task: SonarCloudPrepare@3 # or SonarQubePrepare@3
    inputs:
      SonarCloud: 'sonarcloud'
      organization: 'foo'
      scannerMode: 'dotnet'
      projectKey: 'foo_sonar-scanning-someconsoleapp'
      projectName: 'sonar-scanning-someconsoleapp'
      extraProperties: |
        sonar.verbose=true

Please note that most of the relevant logging will happen during the END step, i.e. in SonarCloudAnalyze, SonarQubeAnalyze or Run Code Analysis.

Share the msbuild detailed logs

MsBuild.exe /t:Rebuild /v:d

or

dotnet build -v:d

Share the binlog

When investigating a performance issue, it is helpful to also have access to the binlog:

MsBuild.exe /p:reportanalyzer=true /bl:build.binlog

or

dotnet build /p:reportanalyzer=true /bl:build.binlog

The parameters for detailed logs and binlogs can be combined.

 
Thx,
Ann

[uchendu.myky (Uch)]

Hi Ann. Thanks for the detailed instructions. We will follow the steps described, once our billing is reset. Then we’ll share the logs with you. We expect this to be on Dec 15.

Best, Uchendu

[ganncamp (G Ann Campbell)]

Hi Uchendu,

Why wait until your billing is reset? You’re seeing the problem now, under current conditions. Let’s look at the logs under those conditions.

 

Ann

[uchendu.myky (Uch)]

Hi Ann,

we do not see the issue right now. As stated at the very beginning:

As a temporary fix, we just upgraded our plan to 300k LoC. The upgrade will end after a month and we have set it back to the original limit of 200k LoC.

However, since I can still share the log, let me do that:

• dotnet build -v:d
  build-log.txt (9.4 KB)

• sonar cloud verbose output

  begin-log.txt (15.2 KB)
  The log of the end statement is not very large, the verbose option was not valid:

  This setting is not valid in the "end" phase in this version of the C# plugin: sonar.verbose

We’d like to not share the binlog publicly. If needed, we’d have to look for a private and secure exchange option.

Just in case that it matters, our full command for the scan is as follows:

- dotnet sonarscanner begin /d:sonar.host.url="https://sonarcloud.io" /k:"org_backend" /o:"org" /d:sonar.scanner.scanAll=false 
- dotnet build --no-incremental org.Backend.sln
- dotnet sonarscanner end

Could it be, that the dotnet build produces largly more LoC, than displayed eventually in the UI about our project in SonarCloud?

Regards,
Uchendu

[ganncamp (G Ann Campbell)]

Hi Uchendu,

So you’re saying you’re not currently blocked? This makes it so much easier!

Okay, before upgrading your license, analysis of the .NET project was failing with ‘too many LOC’. Then you upgraded your license and now analysis succeeds, right? So when you look at that .NET project in SonarQube Cloud… how many LOC do you see when you look at the branch that was previously failing?

 
Ann

[uchendu.myky (Uch)]

Great hint!

Looking at the history of analysis, there is one run where 78k LoC were added:

Now I’m still puzzled:

1. Why are 41k LoC + 60k LoC + 78k LoC too much for a limit of 200k LoC? (i.e. repo 1 + repo 2 + changes on repo 1)
2. Looking at the commit history and commits triggering this analysis, as well as previous and succeeding commits, I can not find anything suspicious:
   1. no files moved
   2. just few files edited
   3. LoC changed according to git < 1k in the three previous and following (squashed) merge commits.
3. Continuing looking through the history of analysis, I can never see a negative LoC count with which we could come down again to 41k LoC on the branch.

Do you have any hints, for what clues I should look out to identify the issue?

And yes, we are currently not blocked.

Best,
Uchendu

[ganncamp (G Ann Campbell)]

Hi Uchendu,

Please go to the measures tab, expand the Size section and tell me how many LoC you see there.

Then go to Code and browse what shows up there. You’re looking for the files / directories you aren’t expecting to see. This will help you craft your exclusions to get your analysis back under control.

 
HTH,
Ann

[uchendu.myky (Uch)]

Hi Ann

The measure tab on our main branch shows:

Browsing through the code shows a total of 41k, which matches the LoC in the measures tab.

Looking at the Merge Request from two weeks ago, where the issue popped up, not much is shown:

The Merge Requests prior to and after the failing one do not seem suspicious.

Best
Uchendu

[ganncamp (G Ann Campbell)]

Hi Uchendu,

Can you (or your SonarQube Cloud admin) go to Billing and Usage and see how many LOC you currently have under analysis?

 
Thx,
Ann

[LuMyky]

Hi Ann,

Sorry for the late response. In the screenshot, you can see our current usage.

Best
Lukas

[ganncamp (G Ann Campbell)]

Hi Lukas,

Welcome to the community!

I take it you work with Uchendu?

Based on all your (combined) screenshots, it looks like you’re good to go right now. Your latest analysis found 41k LOC in that problematic project and your current license uses is just over 100k, as described in the first post.

I can only guess that a library was accidentally added to analysis and then removed.

If you run into this again, turn on debug analysis logging (/d:sonar.verbose=true on the begin command line) to get an exhaustive (truly ) list of the files indexed by analysis to see what extra files are being picked up that shouldn’t be.

 
HTH,
Ann

[LuMyky]

Hi Ann,

Right, I should have mentioned that . We’ll be doing a downgrade to 200k LOC on December 14 and hope that the issue won’t reappear. If it does, we’ll reach out again.

Thanks a lot for your support and for the tip about the debug analysis logging!

Best regards,
Lukas