Error: Google.Protobuf.dll' already exists

  • SonarQube Community Edition Version 8.4.1 buld 35646

  • SonarScanner 4.9.0.17385

I have a pipeline that runs 4 simultaneous builds, sometimes they all work, and sometimes I have this error in one of them:

+ mkdir -p /tmp/QrCodeWebApi

+ HOME=/tmp/QrCodeWebApi

+ dotnet-sonarscanner begin /k:csf-api-conta-digital-qrcode /d:sonar.host.url=https://sonarqube.domain.com /d:sonar.login=2947da4f374026c7263daf67c9f5c82d35940496 /d:sonar.cs.opencover.reportsPaths=/home/jenkins/agent/workspace/conta_digital_api_qrcode/app-bundle-conta-digital-api-qrcode/commit/src/conta_digital_api_qrcode/app-source-conta-digital-api-qrcode/coverage.opencover.xml

SonarScanner for MSBuild 4.9

Using the .NET Core version of the Scanner for MSBuild

Pre-processing started.

Preparing working directories...

17:46:56.506  Updating build integration targets...

17:46:56.544  Fetching analysis configuration settings...

17:47:03.246  Provisioning analyzer assemblies for cs...

17:47:03.247  Installing required Roslyn analyzers...

Unhandled exception. System.IO.IOException: The file '/tmp/.sonarqube/resources/0/Google.Protobuf.dll' already exists.

   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)

   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)

   at System.IO.FileStream.OpenHandle(FileMode mode, FileShare share, FileOptions options)

   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)

   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean useAsync)

   at System.IO.Compression.ZipFileExtensions.ExtractToFile(ZipArchiveEntry source, String destinationFileName, Boolean overwrite)

   at System.IO.Compression.ZipFileExtensions.ExtractRelativeToDirectory(ZipArchiveEntry source, String destinationDirectoryName, Boolean overwrite)

   at System.IO.Compression.ZipFileExtensions.ExtractToDirectory(ZipArchive source, String destinationDirectoryName, Boolean overwriteFiles)

   at System.IO.Compression.ZipFile.ExtractToDirectory(String sourceArchiveFileName, String destinationDirectoryName, Encoding entryNameEncoding, Boolean overwriteFiles)

   at System.IO.Compression.ZipFile.ExtractToDirectory(String sourceArchiveFileName, String destinationDirectoryName)

   at SonarScanner.MSBuild.PreProcessor.Roslyn.EmbeddedAnalyzerInstaller.FetchResourceFromServer(Plugin plugin, String targetDir)

   at SonarScanner.MSBuild.PreProcessor.Roslyn.EmbeddedAnalyzerInstaller.GetPluginResourceFiles(Plugin plugin)

   at SonarScanner.MSBuild.PreProcessor.Roslyn.EmbeddedAnalyzerInstaller.InstallAssemblies(IEnumerable`1 plugins)

   at SonarScanner.MSBuild.PreProcessor.Roslyn.RoslynAnalyzerProvider.FetchAnalyzerPlugins(String language, IEnumerable`1 activeRules)

   at SonarScanner.MSBuild.PreProcessor.Roslyn.RoslynAnalyzerProvider.SetupAnalyzer(TeamBuildSettings teamBuildSettings, IAnalysisPropertyProvider sonarProperties, IEnumerable`1 activeRules, IEnumerable`1 inactiveRules, String language)

   at SonarScanner.MSBuild.PreProcessor.TeamBuildPreProcessor.FetchArgumentsAndRulesets(ISonarQubeServer server, ProcessedArgs args, TeamBuildSettings settings, IDictionary`2& serverSettings, List`1& analyzersSettings)

   at SonarScanner.MSBuild.PreProcessor.TeamBuildPreProcessor.DoExecute(ProcessedArgs localSettings)

   at SonarScanner.MSBuild.PreProcessor.TeamBuildPreProcessor.Execute(String[] args)

   at SonarScanner.MSBuild.BootstrapperClass.PreProcess()

   at SonarScanner.MSBuild.BootstrapperClass.Execute()

   at SonarScanner.MSBuild.Program.Execute(String[] args, ILogger logger)

   at SonarScanner.MSBuild.Program.Execute(String[] args, Boolean showDeprecatedWarning)

   at SonarScanner.MSBuild.Program.Main(String[] args)

/home/jenkins/agent/workspace/conta_digital_api_qrcode/app-bundle-conta-digital-api-qrcode/commit/src/conta_digital_api_qrcode/app-source-conta-digital-api-qrcode@tmp/durable-b8361dec/script.sh: line 1:  1154 Aborted                 (core dumped) HOME=/tmp/QrCodeWebApi dotnet-sonarscanner begin /k:'csf-api-conta-digital-qrcode' /d:sonar.host.url='https://sonarqube.domain.com' /d:sonar.login='2947da4f374026c7263daf67c9f5c82d35940496' /d:sonar.cs.opencover.reportsPaths=/home/jenkins/agent/workspace/conta_digital_api_qrcode/app-bundle-conta-digital-api-qrcode/commit/src/conta_digital_api_qrcode/app-source-conta-digital-api-qrcode/coverage.opencover.xml

script returned exit code 134

Hi Ramon,

Welcome to the community!

Please keep in mind you need to share some mandatory information when posting here:

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
  • what are you trying to achieve
  • what have you tried so far to achieve this

On top of answering those questions, can you clarify whether you are running several SonarScanner processes in parallel on your pipeline? From your pipeline design and the IO related error, I am thinking you may be getting into trouble due to executing 2 or more SonarScanner processes concurrently on the “Commit2” stage which operate on the same path. So, let’s start from there.

Cheers,
Daniel

1 Like

Hi @Daniel_Meppiel, how are you? Thanks a lot for your feedback. @Ramon_Sobreira will provide all information soon, but meanwhile, I would like to give you further information.

Basically, our devsecops framework is split into two parts: COMMIT stage and DEPLOYMENT stage. Now, we are talking about the first one which begins with a change to the state of the project/application —that is, a commit to the version control system, compile code, run unit tests and some integration tests (when it exists), perform quality analysis of the code, prepare artifacts to use in later stage of pipeline – binaries. Include a version number into a bundle of the entire solution. Then, it ends with either a report of failure or, if successful, a collection of binary artifacts and deployable assemblies to be used in subsequent test and release stages, as well as reports on the state of the application.

Particularly, there are repositories in the web-based version control repository hosting service that have more than one project or source code. For this scenario, it is possible to process the steps above in paralell in the same ci/cd pipeline.

At the beginning, that error rarely occurred due to we only had 2 projects and probably because we were in the onboarding phase, but now we have several structures which have 4, 6 projects or so in the same repository, then this error frenquently happens during this process.

Therefore, we would like to know if there is a solution for this problem. Otherwise, what’s the procedure to open an enhancement request to the SonarQube Developers to assess that? And if it is accepted, I don’t have any doubt this may help us and the entire community a lot.

Thank you,
Alexandre RABELO

Hi Alexandre,

Thanks for your message.

So you have a Monorepo?

I am more convinced now that you are running into a “race condition” where multiple SonarQube scans are running in parallel and may be overwriting/reading from the same folder. The scanners needs to create their temporary storage folder under /tmp/.sonarqube, so you need to make sure this location is used by a single scanner once at a time. With 6 scanners running in parallel and all of them using the same temporary folder for their work, anything and everything can happen.

Just so you know, SonarQube does not offer for now a clean/seamless way to manage your scenario as of today, although it is in our vision to provide Monorepo support as SonarCloud already does. So you probably need to work in isolating/disentangling the execution of the 6 scanners or execute a single scan for the entire repository.

Hope it helps, cheers,
Daniel

2 Likes

@Daniel_Meppiel, I’m assessing your return, but I agree with your feedback. We work with BITBUCKET, however, the behaviour that you described matches with the situations that are faced by us.

Thank you for your great support,
Alex

Hello @Daniel_Meppiel, how are you?
@Jefferson_Dias and I identified a great solution that worked very well to our case. Basically, we set TMPDIR up before running the sonarscanner begin and it takes the appropriated directory that is specified by each application so even when they run the processes in pararell, this CLI below works with their respective temporary folder due to this environment variable belongs to the operating system and it seems the sonarscanner uses it instead of working with the file system hard-coded.

TMPDIR=/tmp/${dirName} dotnet-sonarscanner begin /k:’{{ sonar_project_key }}’ /d:sonar.host.url=’{{ sonar_endpoint_url }}’ /d:sonar.login=’{{ sonar_token }}’ /d:sonar.cs.opencover.reportsPaths={{ base_dir }}/coverage.opencover.xml

We used this material to support us in this implementation:

Finally, we would like to suggest if it is possible to update the documentation when there are CI/CD pipelines running in parallel or with concurrent builds, so this can be applicable for several users that are facing the same situation.

Thank you,
Ramon Sobreira
@Jefferson_Dias

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

Hi @Ramon_Sobreira,

I am glad you could find the solution above. And thanks for posting the workaround here, really valuable.

I am now checking internally whether this may be reflecting an issue on the SonarScanner for MSBuild and the best path of action for us. I’ll keep you posted.

Have a great day,
Daniel