Ever since I installed 9.6.1, this is happening for a large project.
I can’t see any memory problems in the logs.
I tried to increase CE to 3G for large projects, but no fix.
Failed to execute task AYNbxLF4GxLJ899PdyK6
org.sonar.ce.task.projectanalysis.component.VisitException: Visit failed for Component
{key=ross_dashboard_dashboard_AYLWjHYd7yd4l8x8fkgH:
***.java,type=FILE}
located ***(
type=DIRECTORY)->***(
type=DIRECTORY)->***(
type=DIRECTORY)->***(
type=DIRECTORY)->ross_dashboard_dashboard_AYLWjHYd7yd4l8x8fkgH(type=PROJECT)
at org.sonar.ce.task.projectanalysis.component.VisitException.rethrowOrWrap(VisitException.java:44)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visit(PathAwareCrawler.java:52)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitChildren(PathAwareCrawler.java:87)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitImpl(PathAwareCrawler.java:70)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visit(PathAwareCrawler.java:50)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitChildren(PathAwareCrawler.java:87)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitImpl(PathAwareCrawler.java:70)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visit(PathAwareCrawler.java:50)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitChildren(PathAwareCrawler.java:87)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitImpl(PathAwareCrawler.java:70)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visit(PathAwareCrawler.java:50)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitChildren(PathAwareCrawler.java:87)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitImpl(PathAwareCrawler.java:70)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visit(PathAwareCrawler.java:50)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitChildren(PathAwareCrawler.java:87)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitImpl(PathAwareCrawler.java:70)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visit(PathAwareCrawler.java:50)
at org.sonar.ce.task.projectanalysis.step.NewCoverageMeasuresStep.execute(NewCoverageMeasuresStep.java:90)
at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:80)
at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:71)
at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:58)
at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:75)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:212)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:194)
at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:160)
at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:135)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:87)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NullPointerException: null
at org.sonar.ce.task.projectanalysis.source.NewLinesRepository.computeNewLinesFromScm(NewLinesRepository.java:84)
at org.sonar.ce.task.projectanalysis.source.NewLinesRepository.getNewLines(NewLinesRepository.java:64)
at org.sonar.ce.task.projectanalysis.step.NewCoverageMeasuresStep$NewCoverageCounter.initialize(NewCoverageMeasuresStep.java:205)
at org.sonar.ce.task.projectanalysis.formula.FormulaExecutorComponentVisitor.processLeaf(FormulaExecutorComponentVisitor.java:148)
at org.sonar.ce.task.projectanalysis.formula.FormulaExecutorComponentVisitor.process(FormulaExecutorComponentVisitor.java:125)
at org.sonar.ce.task.projectanalysis.formula.FormulaExecutorComponentVisitor.visitFile(FormulaExecutorComponentVisitor.java:105)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitNode(PathAwareCrawler.java:102)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitImpl(PathAwareCrawler.java:73)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visit(PathAwareCrawler.java:50)
Could you share what type of new code period strategy are you using in SQ? Are you using git? Could you also share scanner execution logs from that failure?
Hey I am also getting similar error.
SonarQube server 9.6.1.59531, SonarScanner 4.7.0.2747. SonarQube Code Analysis never runs on my github project as the analysis is failing in the sonar server for the PRs.
org.sonar.ce.task.projectanalysis.component.VisitException: Visit failed for Component {key=MyProject-web:src/components/comp.tsx:PULL_REQUEST:10,type=FILE} located MyProject-web:src/components/comp:1052(type=DIRECTORY)->MyProject:PULL_REQUEST:10(type=PROJECT)
at org.sonar.ce.task.projectanalysis.component.VisitException.rethrowOrWrap(VisitException.java:44)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visit(PathAwareCrawler.java:52)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitChildren(PathAwareCrawler.java:87)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitImpl(PathAwareCrawler.java:70)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visit(PathAwareCrawler.java:50)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitChildren(PathAwareCrawler.java:87)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitImpl(PathAwareCrawler.java:70)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visit(PathAwareCrawler.java:50)
at org.sonar.ce.task.projectanalysis.step.NewCoverageMeasuresStep.execute(NewCoverageMeasuresStep.java:90)
at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:80)
at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:71)
at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:58)
at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:75)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:212)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:194)
at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:160)
at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:135)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:87)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NullPointerException
at org.sonar.ce.task.projectanalysis.scm.ScmInfoRepositoryImpl.removeAuthorAndRevision(ScmInfoRepositoryImpl.java:104)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:550)
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:517)
at org.sonar.ce.task.projectanalysis.scm.ScmInfoRepositoryImpl.removeAuthorAndRevision(ScmInfoRepositoryImpl.java:99)
at org.sonar.ce.task.projectanalysis.scm.ScmInfoRepositoryImpl.generateAndMergeDb(ScmInfoRepositoryImpl.java:119)
at org.sonar.ce.task.projectanalysis.scm.ScmInfoRepositoryImpl.getScmInfoForComponent(ScmInfoRepositoryImpl.java:74)
at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1134)
at org.sonar.ce.task.projectanalysis.scm.ScmInfoRepositoryImpl.getScmInfo(ScmInfoRepositoryImpl.java:65)
at org.sonar.ce.task.projectanalysis.source.NewLinesRepository.computeNewLinesFromScm(NewLinesRepository.java:72)
at org.sonar.ce.task.projectanalysis.source.NewLinesRepository.getNewLines(NewLinesRepository.java:64)
at org.sonar.ce.task.projectanalysis.step.NewCoverageMeasuresStep$NewCoverageCounter.initialize(NewCoverageMeasuresStep.java:205)
at org.sonar.ce.task.projectanalysis.formula.FormulaExecutorComponentVisitor.processLeaf(FormulaExecutorComponentVisitor.java:148)
at org.sonar.ce.task.projectanalysis.formula.FormulaExecutorComponentVisitor.process(FormulaExecutorComponentVisitor.java:125)
at org.sonar.ce.task.projectanalysis.formula.FormulaExecutorComponentVisitor.visitFile(FormulaExecutorComponentVisitor.java:105)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitNode(PathAwareCrawler.java:102)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visitImpl(PathAwareCrawler.java:73)
at org.sonar.ce.task.projectanalysis.component.PathAwareCrawler.visit(PathAwareCrawler.java:50)
... 26 more
We have the same setup as Gaurav with the Enterprise Edition and the same null pointer exception.
Please advise how to resolve.
Error Details
org.sonar.ce.task.projectanalysis.component.VisitException: Visit of Component {key=Vitality:27c98c29:pom.xml:BRANCH:feature/testing,type=FILE} failed
at org.sonar.ce.task.projectanalysis.component.VisitException.rethrowOrWrap(VisitException.java:44)
at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visit(VisitorsCrawler.java:71)
at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visitChildren(VisitorsCrawler.java:107)
at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visitImpl(VisitorsCrawler.java:94)
at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visit(VisitorsCrawler.java:69)
at org.sonar.ce.task.projectanalysis.step.ExecuteVisitorsStep.execute(ExecuteVisitorsStep.java:51)
at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:80)
at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:71)
at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:58)
at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:75)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:212)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:194)
at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:160)
at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:135)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:87)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NullPointerException
at org.sonar.ce.task.projectanalysis.scm.ScmInfoRepositoryImpl.removeAuthorAndRevision(ScmInfoRepositoryImpl.java:104)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:550)
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:517)
at org.sonar.ce.task.projectanalysis.scm.ScmInfoRepositoryImpl.removeAuthorAndRevision(ScmInfoRepositoryImpl.java:99)
at org.sonar.ce.task.projectanalysis.scm.ScmInfoRepositoryImpl.generateAndMergeDb(ScmInfoRepositoryImpl.java:119)
at org.sonar.ce.task.projectanalysis.scm.ScmInfoRepositoryImpl.getScmInfoForComponent(ScmInfoRepositoryImpl.java:74)
at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1134)
at org.sonar.ce.task.projectanalysis.scm.ScmInfoRepositoryImpl.getScmInfo(ScmInfoRepositoryImpl.java:65)
at org.sonar.ce.task.projectanalysis.source.LastCommitVisitor.visitFile(LastCommitVisitor.java:76)
at org.sonar.ce.task.projectanalysis.component.PathAwareVisitorWrapper.visitFile(PathAwareVisitorWrapper.java:61)
at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visitNode(VisitorsCrawler.java:123)
at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visitImpl(VisitorsCrawler.java:97)
at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visit(VisitorsCrawler.java:69)
... 23 more
We have had 3 more instances of this bug in 3 separate repositories today. This is not caused by low disk space as previously suspected. Once it starts then it happens to every subsequent branch analysis on the project and can only be fixed by deleting the entire project, losing all the analysis history for all branches.
The Jira notes you link to above imply missing SCM data - we use Bitbucket Server 7.17.10
Is there a workaround until the 9.8 fix is released that doesn’t involve removing the entire analysis for all branches and losing the analysis history or a project?
We have tried:
a) removing the branch analysis
b) resetting the Devops Platform Integration setting
c) removing an entire project
Only c) works at this stage.
Ideally there would be a way to force the refresh of the Sonar SCM data for a project.
could you please try to scan with sonar.scm.forceReloadAll=true?
If the analysis of new branches are failing, try to analyze the main branch of the project first, using that flag.
That is a setting applied always to all analyses we do. I have checked that earlier and also tried runs disabling the SCM sensor for the project, running an analysis (no improvement) then enabling it again and repeating. Also a restart of the server did not resolve it and when it occurred a couple of weeks ago clearing the cache entirely and waiting for all projects to reload also did not work.
Do you have any other suggestions?
When is 9.8 to be released?
9.8 is planned to be released in a few weeks, the beginning of December.
The scanner might be failing to collect blame information for certain files. Do you have any relevant logs about that on the scanner? Might be worth enabling debug logs.
02:53:42.1926647 - [50742] - 02:53:42.130 DEBUG: Unable to blame file src/Feature/Rules/code/Conditions/StatusCondition.cs. No blame info at line 1. Is file committed? [Author: PersonIdent[Not Committed Yet, , Thu Nov 17 02:53:42 2022 +0000] Source commit: null]
02:53:42.6458126 - [50758] - 02:53:42.520 DEBUG: Unable to blame file src/Foundation/Services/code/DocumentUploadService.cs. No blame info at line 1. Is file committed? [Author: PersonIdent[Not Committed Yet, , Thu Nov 17 02:53:42 2022 +0000] Source commit: null]
02:53:42.8489513 - [50775] - 02:53:42.833 DEBUG: Unable to blame file src/Feature/SeeBestNextAction/code/Models/SeeBestNextActionViewModel.cs. No blame info at line 1. Is file committed? [Author: PersonIdent[Not Committed Yet, , Thu Nov 17 02:53:42 2022 +0000] Source commit: null]
That’s part of the problem. The scanner should be able to collect blame for all lines.
Is it possible that there are uncommitted changes to those files when the scanner runs?
I don’t think that is the case. This was a fresh checkout from Bitbucket of the latest commit on the branch.
If the project is deleted and started from fresh the problem goes away. Is there a way to force the cache to refresh for the one project affected without deleting it and starting it again? This particular repository analysis has a long history we wish to retain.
An analysis on the same repository/branch has just been performed using the same scanner and settings but pushing the results to the staging sonar server, which is identical to live. This completes fine but the same blame related errors like those above are the only unusual entries in the log. eg.
11:55:16.4231599 - [60055] - 11:55:13.626 DEBUG: Unable to blame file src/Feature/Rules/code/Conditions/VitalityStatusCondition.cs. No blame info at line 1. Is file committed? [Author: PersonIdent[Not Committed Yet, , Thu Nov 17 11:55:13 2022 +0000] Source commit: null]
I understand, the problem is that when the blame info is not collected from the SCM in the scanner, SonarQube tries to reuse data that is already in the DB. Due to the bug that will be fixed in 9.8, the process to reuse that information fails if the blame data is incomplete in the DB, which looks like is the case for the failing project.
Unfortunately there’s no way to delete the blame data in the DB.
One solution is to understand why the scanner is not collecting blame for those files, and try to fix it.
Another solution would be to temporarily exclude those files in one scan, which would probably cause the server to delete the SCM blame info for those files and hopefully solve the problem.
Thanks for your attention on this. Your explanations have been most helpful.
Thus we’ve managed to resolve this by forcing git to (re)copy from index to the work-tree to remove any uncommitted changes to tracked files just before the analysis step using:
git checkout -- .
Although no visible changes to files, something in the pipeline was touching files enough to create uncommitted changes at the top of some files. This command resets those changes to tracked files without losing important analysis artifacts.