Issues setting up Azure Devops NX monorepo

Last week I’ve attempted to setup SonarCloud for a Nx repository hosted in Azure DevOps.
However, I’m having some trouble understand the functioning of the Azure DevOps SonarCloud prepare step and, the monorepo setup in general.

For clarification we have both .NET Core applications, libraries and, Angular applications and libraries.
Generally speaking a NX monorepo looks like this:

apps/
├─ app-1/
libs/
├─ lib-1/

In the azure-pipelines.yml:

  • I’ve added a SonarCloudPrepare task for each application and library.
  • I’ve added keys to each SonarCloudPrepare task.
  • I’ve set the sonar.projectVersion to $(Build.SourceVersion) for each SonarCloudPrepare task. (We don’t have version management that uses semantic versioning or any kind of tagging)
  • I’ve added a SonarCloudAnalyze task.
  • I’ve added a SonarCloudPublish task.

In SonarCloud:

  • I’ve enabled Monorepo support for each project.
  • I’ve set the analysis scope and narrowed the inclusion to: “apps/app-x//*" or "libs/lib-X//*”.
  • I’ve set the New Code Definition to “Previous version”.

This “seems” to go well. However, when I check the Analysis logs and the Publication on SonarCloud I only ever see one project being updated with a new analysis.
This project is always the last project for which a SonarCloudPrepare task was started.

I’m quite baffled by this behavior and I would love some support here.

Thanks in advance, Rope.

Is there another way for me to get in contact with SonarSource about this?
If possible I’d like to fix this by the end of the week.

Hey there.

It might be easiest if you share your azure-pipelines.yml here so someone can take a look at where there might be an issue.

Hi Colin,

Thanks for your reply. I’ve added the relevant sections below.

steps:

  - script: npx ts-node ./set-affected-projects-variable.ts $(nxFlags.NxAffectedScopeFlag)
    displayName: 'Set Affected Projects variable for SonarCloudPrepare'

  - task: SonarCloudPrepare@1
    condition: contains(variables.NxAffectedProjects, 'api1')
    inputs:
      SonarCloud: 'SonarInstanceName'
      organization: 'organization'
      scannerMode: 'MSBuild'
      projectKey: 'key-of-api-1'
      projectName: 'Description of API 1'
      projectVersion: $(Build.SourceVersion)

  - task: SonarCloudPrepare@1
    condition: contains(variables.NxAffectedProjects, 'api2')
    inputs:
      SonarCloud: 'SonarInstanceName'
      organization: 'organization'
      scannerMode: 'MSBuild'
      projectKey: 'key-of-api-2'
      projectName: 'Description of API 2'
      projectVersion: $(Build.SourceVersion)

  - task: SonarCloudPrepare@1
    condition: contains(variables.NxAffectedProjects, 'api3')
    inputs:
      SonarCloud: 'SonarInstanceName'
      organization: 'organization'
      scannerMode: 'MSBuild'
      projectKey: 'key-of-api-3'
      projectName: 'Description of API 3'
      projectVersion: $(Build.SourceVersion)

  - task: SonarCloudPrepare@1
    condition: contains(variables.NxAffectedProjects, 'app1')
    inputs:
      SonarCloud: 'SonarInstanceName'
      organization: 'organization'
      scannerMode: 'MSBuild'
      projectKey: 'key-of-app-1'
      projectName: 'Description of APP 1'
      projectVersion: $(Build.SourceVersion)

  - script: npx nx affected $(nxFlags.NxAffectedScopeFlag) --target=lint --parallel --max-parallel=3
    displayName: 'Lint'

  # Running tests in parallel seems to become flaky...
  - script: npx nx affected $(nxFlags.NxAffectedScopeFlag) --target=test --configuration=ci
    displayName: 'Test'

  - script: npx nx affected $(nxFlags.NxAffectedScopeFlag) --target=e2e --configuration=ci
    displayName: 'e2e'

  - script: npx nx affected $(nxFlags.NxAffectedScopeFlag) --target=build --parallel --max-parallel=3 --skip-nx-cache
    displayName: 'Build'

  - script: npx nx affected $(nxFlags.NxAffectedScopeFlag) --target=post-build --parallel --max-parallel=3
    displayName: 'Post-Build'

  - task: SonarCloudAnalyze@1
    continueOnError: true

  - task: SonarCloudPublish@1
    continueOnError: true
    inputs:
      pollingTimeoutSec: '300'

Then in the console output logs of the build I see this log:

INFO: Load project settings for component key: 'app1'
INFO: Load project settings for component key: 'app1' (done) | time=32ms

There are no other logs with this message for any of the other projects.
Furthermore the logs contain messages:

INFO: Indexing files of module 'Api3'
INFO:   Base dir: /home/vsts/work/1/s/apps/api3
INFO:   Source paths: obj/Debug/net6.0/apphost
INFO:   Included sources: apps/app1
INFO:   Excluded sources: **/build-wrapper-dump.json

For each of the .NET Core applications that are in the mono-repo.

Then there are warning messages in the logs for each file of every project of this type:

WARN: File '/home/vsts/work/1/s/apps/api3/Controllers/SomeController.cs' referenced by the protobuf 'MetricsInfo' does not exist in the analysis context

Which is expected because the files are not included in the app1 project settings.
What is not expected is that the modules are scanned at all.

Hey there.

Just stating upfront – no experience with Nx here other than a quick glance at the documentation.

To accomplish what I think you’re trying to do (separate projects for each build), you would need to wrap each individual project build with its own Prepare and Analyze step. We also don’t well support a setup where projects are selectively built based on what changed (and for C# code – we can’t get analysis results without a build).

Can you tell us a bit more about what Nx is doing behind the scenes? Is it, for example, not running a dotnet build on a project if it hasn’t changed?

To accomplish what I think you’re trying to do (separate projects for each build)

Not entirely. Separate projects for each application. It is intended to be setup as stated in the SonarCloud documentation. [ Monorepo support & SonarCloud ].
I was unable to find a mono-repo example hence, I assumed this is what the documentation intended.

Is it, for example, not running a dotnet build on a project if it hasn’t changed?

This is correct. Nx can retrieve the applications/libraries affected by a change and run selected tasks on them.
This is why I added conditions to prevent SonarCloudPrepare from being run for projects which were not affected by a change.

Hi Colin,

Did you have a chance to look at this a bit more?
Or should I create a bug-ticket?

Hi Colin,

Bumping this topic so I know whether I should createa bug-ticket or not.

Hey @Rope

This is a community forum where we try and help users get the most our of Sonar. We also take holidays, pay attention to other threads, etc. Your patience is appreciated, but bumping threads multiple threads is not.

Thanks for the further details.

SonarCloud relies on a pair of SonarCloudPrepare and SonarCloudAnalyze for each project being targeted for analysis. A single SonarCloudAnalyze will not publish results to multiple SonarQube projects.

Today, I don’t think there’s any graceful way to handle a situation where what is being built by a single build command changes from build to build.

You would effectively need to change your build to have multiple build commands each wrapped by a different set of Prepare/Analayze tasks (which only build one specific project), which I suppose you’d only want to run if they are affected.

I’m happy to flag this thread for some expert attention in case our .NET experts have some other ideas (or simply to make them aware that this tooling is out there, and we don’t integrate well with it)

From my POV this is not entirely correct. Since SonarCloudAnalyze does not define input variables which would allow providing a project-key. Hence the pairing is not explicit but implicit. This means, as you stated, the only way to analyze a build is to run a SonarCloudPrepare, build a single application and then run a SonarCloudAnalyze.

Furthermore, a consequence is that any preceding SonarCloudAnalyze tasks are ignored. This is not stated in the documentation afaik.

Correct. Unfortunately, we cannot run separate tasks to do this because we’re running a mono-repo build-tool which builds multiple applications. I was not able to derive this information from the documentation provided on sonarcloud.io and I would like to suggest this be clarified if possible.

This would be much appreciated!

For those interested.
I ended up writing 3 Nx Custom Executors.
Respectively, 2 for Sonar (prepare and analyze) and 1 to wrap the nx-dotnet build executor with the 2 Sonar executors.

It was quite the hassle to achieve this and I will most likely be filing 2 feature requests.
One with SonarSource to support building multiple projects without having to wrap each dotnet build call and one with nx-dotnet to support dotnet tools tasks.

Marking this reply as the solution.

1 Like

I’ve created the feature-requests I mentioned earlier.

Nx-DotNet: [Feature] Add support for Custom Tools Executors · Issue #455 · nx-dotnet/nx-dotnet · GitHub
SonarSource: Support monorepo build-systems for .NET SonarScanner/Azure Pipelines tasks

Thank you @Rope.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.