NullpointerException during background task analysis for other portfolio

Hi,

We use Sonar enterprise version 8.9.6 running through the docker image distribution in a kubernetes environnement.

We have some portofolios for all of our enterprise teams to organize hundreds of projects.
The “Other” portfolios containing all projects that do not match any project through rules by names or regular expressions, is always failing its computation.
So our “other” portfolio do not display correctly and contains many projects that are correctly matched by other portfolios. Every portfolio computation is correct except the “other” one.

There are 2 bugs here :
1/ In the UI of Sonar listing background tasks, the failing task display a FAILED status but do not have the wheel to show the logs explaining the error.


(nothing on the right of the line)

2/ After digging the logs in the stdout of the sonar container, we can find the exception and the full log of the task :

2022.12.30 14:06:36 INFO ce[][o.s.c.t.CeWorkerImpl] Execute task | project=autre | type=VIEW_REFRESH | id=AYVjJDkIJZm_vk7Vvo1u
2022.12.30 14:06:36 INFO ce[AYVjJDkIJZm_vk7Vvo1u][o.s.c.t.s.ComputationStepExecutor] Initialize analysis | status=SUCCESS | time=1ms
2022.12.30 14:06:36 INFO ce[AYVjJDkIJZm_vk7Vvo1u][o.s.c.t.s.ComputationStepExecutor] Initialize | status=SUCCESS | time=5ms
2022.12.30 14:06:36 INFO ce[AYVjJDkIJZm_vk7Vvo1u][o.s.c.t.s.ComputationStepExecutor] Purge removed views from database | status=SUCCESS | time=7ms
2022.12.30 14:06:36 INFO ce[AYVjJDkIJZm_vk7Vvo1u][o.s.c.t.s.ComputationStepExecutor] Loads View definition | status=SUCCESS | time=7ms
2022.12.30 14:06:36 INFO ce[AYVjJDkIJZm_vk7Vvo1u][o.s.c.t.s.ComputationStepExecutor] Resolve referenced projects on Portofolio | status=SUCCESS | time=46ms
2022.12.30 14:06:36 INFO ce[AYVjJDkIJZm_vk7Vvo1u][o.s.c.t.s.ComputationStepExecutor] Set portfolio refresh state to 'COMPUTING' | status=SUCCESS | time=3ms
2022.12.30 14:06:36 INFO ce[AYVjJDkIJZm_vk7Vvo1u][o.s.c.t.s.ComputationStepExecutor] Retrieve project analyses | status=SUCCESS | time=351ms
2022.12.30 14:06:36 INFO ce[AYVjJDkIJZm_vk7Vvo1u][o.s.c.t.s.ComputationStepExecutor] Creates component tree | status=FAILED | time=18ms
2022.12.30 14:06:36 ERROR ce[AYVjJDkIJZm_vk7Vvo1u][o.s.c.t.CeWorkerImpl] Failed to execute task AYVjJDkIJZm_vk7Vvo1u
java.lang.NullPointerException: null
at com.sonar.governance.ce.G.A$_A.C(Unknown Source)
at com.sonar.governance.ce.G.A$_A.E(Unknown Source)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(Unknown Source)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
at com.sonar.governance.ce.G.A$_A.A(Unknown Source)
at com.sonar.governance.ce.G.A$_A.E(Unknown Source)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(Unknown Source)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
at com.sonar.governance.ce.G.A$_A.A(Unknown Source)
at com.sonar.governance.ce.G.A$_A.A(Unknown Source)
at com.sonar.governance.ce.G.A.execute(Unknown Source)
at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:81)
at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:72)
at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:59)
at com.sonar.governance.ce.J.A.A.B.A(Unknown Source)
at com.sonar.governance.ce.J.A.A.F.A(Unknown Source)
at com.sonar.governance.ce.C.A.B(Unknown Source)
at com.sonar.governance.ce.C.A.process(Unknown Source)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:212)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:194)
at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:160)
at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:135)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:87)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
2022.12.30 14:06:36 INFO ce[AYVjJDkIJZm_vk7Vvo1u][o.s.c.t.CeWorkerImpl] Executed task | project=autre | type=VIEW_REFRESH | id=AYVjJDkIJZm_vk7Vvo1u | status=FAILED | time=475ms

This error should be corrected or better manager in the sonar engine.
But for now, can you give us a clue where the problem is on of the portfolio parameterization or somewhere else ?

Thanks,

Hello @nico7 ,

Could you please clarify what you mean by

The “Other” portfolios containing all projects that do not match any project through rules by names or regular expressions, is always failing its computation.

If you select the option “All projects” in the portfolio definition, all projects of SonarQube will be added to this portfolio, regardless if it is already in another portfolio or not. Did you define a specific rule for that?
Could you show us the portfolio definition?

Regarding 1/, do you see the wheel when the portfolio recomputation is working? For another task type (Project Analysis for instance)?

Thanks

Hi @leo.geoffroy,
Yes, by “Other” portfolios, it is the option selection of “All projects” and “All remaining Projects” in the definition.
Here is a screen shot :

For the wheel, as the task duration is less than half a second, I do not have seen the wheel during recomputation time. I have only seen the line when the task is finished.

Thanks for your help on this,

Hello @nico7,

Could you try to use the “All remaining projects” in a sub-portfolio, and not on the parent portfolio as it is shown on your screenshot.

I believe the All Remaining Projects only applies for the projects left at the same level of portfolios, And since “Projets Orphelins” is one level higher, it shows all the projects regardless of how other portfolios are configured

Hi @leo.geoffroy ,
I did what you suggest but the problem is the same :

And the logs produced by Sonar are nearly the same :

2023.01.10 15:30:44 INFO ce[][o.s.c.t.CeWorkerImpl] Execute task | project=autre | type=VIEW_REFRESH | id=AYWcFzGXC6y656hkh992 | submitter=xxx54745
2023.01.10 15:30:44 INFO ce[AYWcFzGXC6y656hkh992][o.s.c.t.s.ComputationStepExecutor] Initialize analysis | status=SUCCESS | time=2ms
2023.01.10 15:30:44 INFO ce[AYWcFzGXC6y656hkh992][o.s.c.t.s.ComputationStepExecutor] Initialize | status=SUCCESS | time=5ms
2023.01.10 15:30:44 INFO ce[AYWcFzGXC6y656hkh992][o.s.c.t.s.ComputationStepExecutor] Purge removed views from database | status=SUCCESS | time=11ms
2023.01.10 15:30:44 INFO ce[AYWcFzGXC6y656hkh992][o.s.c.t.s.ComputationStepExecutor] Loads View definition | status=SUCCESS | time=5ms
2023.01.10 15:30:44 INFO ce[AYWcFzGXC6y656hkh992][o.s.c.t.s.ComputationStepExecutor] Resolve referenced projects on Portofolio | status=SUCCESS | time=50ms
2023.01.10 15:30:44 INFO ce[AYWcFzGXC6y656hkh992][o.s.c.t.s.ComputationStepExecutor] Set portfolio refresh state to 'COMPUTING' | status=SUCCESS | time=5ms
2023.01.10 15:30:44 INFO ce[AYWcFzGXC6y656hkh992][o.s.c.t.s.ComputationStepExecutor] Retrieve project analyses | status=SUCCESS | time=260ms
2023.01.10 15:30:45 INFO ce[AYWcFzGXC6y656hkh992][o.s.c.t.s.ComputationStepExecutor] Creates component tree | status=FAILED | time=66ms
2023.01.10 15:30:45 ERROR ce[AYWcFzGXC6y656hkh992][o.s.c.t.CeWorkerImpl] Failed to execute task AYWcFzGXC6y656hkh992
java.lang.NullPointerException: null
at com.sonar.governance.ce.G.A$_A.C(Unknown Source)
at com.sonar.governance.ce.G.A$_A.E(Unknown Source)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(Unknown Source)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
at com.sonar.governance.ce.G.A$_A.A(Unknown Source)
at com.sonar.governance.ce.G.A$_A.E(Unknown Source)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(Unknown Source)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
at com.sonar.governance.ce.G.A$_A.A(Unknown Source)
at com.sonar.governance.ce.G.A$_A.A(Unknown Source)
at com.sonar.governance.ce.G.A.execute(Unknown Source)
at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:81)
at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:72)
at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:59)
at com.sonar.governance.ce.J.A.A.B.A(Unknown Source)
at com.sonar.governance.ce.J.A.A.F.A(Unknown Source)
at com.sonar.governance.ce.C.A.B(Unknown Source)
at com.sonar.governance.ce.C.A.process(Unknown Source)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:212)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:194)
at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:160)
at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:135)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:87)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
2023.01.10 15:30:45 INFO ce[AYWcFzGXC6y656hkh992][o.s.c.t.CeWorkerImpl] Executed task | project=autre | type=VIEW_REFRESH | id=AYWcFzGXC6y656hkh992 | submitter=xxx54745 | status=FAILED | time=419ms 

Hello @nico7

The error is related to this ticket, but it was fixed for version 9.5 and upward.

The issue raises when you try to compute a portfolio that has a local reference to another portfolio that contains itself a subporfolio that has not been computed yet.

So in your example, you can try to compute individually each portfolio defined as local reference : “Direction Infrastructure Matérielle et Document”, “Direction Infrastructure Logicielle”, etc. then only after compute this parent portfolio

Hello @leo.geoffroy ,
I recomputed all the portfolios and the task execution is now successfull.

Thanks a lot for your help.

2 Likes

Happy that it worked!

Regarding the issue with the wheel that doesn’t appear in your case, I have created a ticket on our side.
The issue is very specific to the exception that you got