Background tasks not progressing after migration

SonarQube
,
1

Hi,

We’ve migrated our SonarQube instance from 6.7.1 Community Edition to 7.7 Developer Edition.

We launch the scans of (small) projects from Jenkins and the background tasks are correctly scheduled. However, they never finish and the progress is not visible in the logs:

ce[][o.s.c.t.CeWorkerImpl] Execute task | project=...:common-core-base | type=REPORT | id=AWt1irB03mvo8GHfZtS8 | submitter=jenkins
ce[AWt1irB03mvo8GHfZtS8][o.s.c.t.s.ComputationStepExecutor] Extract report | status=SUCCESS | time=96ms
ce[AWt1irB03mvo8GHfZtS8][o.s.c.t.s.ComputationStepExecutor] Persist scanner context | status=SUCCESS | time=10ms
ce[AWt1irB03mvo8GHfZtS8][o.s.c.t.s.ComputationStepExecutor] Propagate analysis warnings from scanner report | status=SUCCESS | time=21ms
ce[AWt1irB03mvo8GHfZtS8][o.s.c.t.s.ComputationStepExecutor] Execute DB migrations for current project | status=SUCCESS | time=4ms
ce[AWt1irB03mvo8GHfZtS8][o.s.c.t.s.ComputationStepExecutor] Generate analysis UUID | status=SUCCESS | time=0ms
ce[AWt1irB03mvo8GHfZtS8][o.s.c.t.s.ComputationStepExecutor] Load analysis metadata | status=SUCCESS | time=81ms
ce[AWt1irB03mvo8GHfZtS8][o.s.c.t.s.ComputationStepExecutor] Initialize | status=SUCCESS | time=15ms
ce[AWt1irB03mvo8GHfZtS8][o.s.c.t.s.ComputationStepExecutor] Verify billing | status=SUCCESS | time=0ms

No error can be seen in the logs.

Before the migration, everything was fine. We’re using a Postgres hosted in AWS RDS, as before.

Any help would be welcome,
thanks,
Damien

2

First scan actually completed in 1 hour and 14 hours, for ~ 7K lines of code.

A second scan completes in 7 seconds…

3

Hi Damien,

First, sorry for the delay in responding. I’m glad it’s working for you now(?).

I’m not saying nothing’s wrong on the SonarQube side but your experience doesn’t ring any bells for me. I would actually first look at whether the experience is reproducible, and if it’s not or it’s only reproducible intermittently I’d look at what else is going on in the environment (resource constrains in network, db, …)

I know this isn’t very helpful. I’m happy to help go further if we can identify something more concrete.

 
Ann

4

Hi,

The issue has not come back, but for the fact that sometimes, the background task takes more time to complete than in average.

We use a lot of monitoring for the tools in our CI/CD ecosystem. Is there a way to make SonarQube export stats about background tasks processing in Prometheus for example?

Thanks,
Damien

5

Hi Damien,

please take a look at the api api/ce/activity, which return the background tasks queued in the compute engine. You can filter by type=REPORT and status=SUCCESS and look for the executionTimeMs.

Maybe you can use this to feed Prometheus ?

1 Like
6

Hi Pierre,

This looks already promising. Now, one would have to code a specific Prometheus scrapper to convert this.

There is https://github.com/dmeiners88/sonarqube-prometheus-exporter but it does support only scan results metrics, nothing about the “ops” metrics.

To be followed :slight_smile:

Thanks,
Damien