How to add additional libraries to the SonarQube classpath?

Hello,
is there any easy and supported way how to add additional libraries (jar-s) to the CE and WEB servers classpaths?

I’m trying to setup the PostgreSQL JDBC connection via the UNIX socket as described at
https://jdbc.postgresql.org/documentation/use/#unix-sockets
https://kohlschutter.github.io/junixsocket/dependency.html#postgresql
in order to improve performance (see below for some results)

It’s about adding 2 jars to the classpath and modifying the JDBC connect string.
This can work, but there doesn’t appear to be any easy way how to add those 2 jars to the classpath.

I tried placing those jar files into the lib/jdbc/postgresql/ directory, but the server refuses to start with multiple files located there.

Since the -cp java argument can’t be used multiple times in the java arguments (only its last occurrence has an effect) and it’s constructed by your code/scripts it can’t be used in the sonar.web.javaAdditionalOpts and sonar.ce.javaAdditionalOpts.

So far I ended up using the -Xbootclasspath/a java argument in sonar.web.javaAdditionalOpts and sonar.ce.javaAdditionalOpts which works, but I don’t think it’s optimal so I consider this as a workaround.

Is there any better way how to achieve this?

For reference I’m adding some performance comparison data, 2nd lines are for connections via UNIX socket. You can see that the difference is really huge so it’s definitely worth it (if the DB is running at the same host).

  • Persist live measures step times (the most problematic step in our case)
    2022.10.24 09:44:46 INFO ce[abc][o.s.c.t.s.ComputationStepExecutor] Persist live measures | insertsOrUpdates=449088 | status=SUCCESS | time=151333ms
    2022.10.24 17:50:37 INFO ce[xyz][o.s.c.t.s.ComputationStepExecutor] Persist live measures | insertsOrUpdates=449088 | status=SUCCESS | time=28891ms

  • Total task times
    2022.10.24 09:44:55 INFO ce[abc][o.s.c.t.CeWorkerImpl] Executed task | project=be:develop | type=REPORT | branch=develop | branchType=BRANCH | id=abc | submitter=sonarqube | status=SUCCESS | time=178371ms
    2022.10.24 17:50:48 INFO ce[xyz][o.s.c.t.CeWorkerImpl] Executed task | project=be:develop | type=REPORT | branch=develop | branchType=BRANCH | id=xyz | submitter=sonarqube | status=SUCCESS | time=61688ms

By the way I was quite surprised by the difference. It’s way bigger than I’d expect. Couldn’t there be any issue with the connection handling (pooling) at the SonarQube side?

UPDATE: I just looked at the PostgreSQL JDBC driver docs and have performed some additional tests and found out why is the performance via TCP so bad: The SonarQube 8.9.10 (LTS) uses the JDBC driver 42.3.3 which has the tcpNoDelay option accidentally disabled by default. It was reenabled in 43.3.5:
https://jdbc.postgresql.org/changelogs/2022-05-04-42.3.5-release/

With the default config the performance is as bad as shown above. With tcpNoDelay=true the performance is on par with the UNIX socket (see below). So I’d say that the PostgreSQL JDBC driver should be updated in the SonarQube LTS - to at least 42.3.5, or rather 42.3.6.

2022.10.24 18:49:38 INFO ce[def][o.s.c.t.s.ComputationStepExecutor] Persist live measures | insertsOrUpdates=449088 | status=SUCCESS | time=28759ms

2022.10.24 18:49:47 INFO ce[def][o.s.c.t.CeWorkerImpl] Executed task | project=be:develop | type=REPORT | branch=develop | branchType=BRANCH | id=def | submitter=sonarqube | status=SUCCESS | time=59067ms

SonarQube 8.9.10 LTS at Java 11

1 Like

Hey, thank you so much for taking the time to report this and putting effort into the investigation. This kind of contribution really helps us improve SonarQube :+1:

I won’t enter much into details about the idea of adding additional libraries to SQ: I don’t see a clear use case, and as you found out, the root cause of your issue is a bug in the PostgreSQL driver.

We are going to run a benchmark on our side to confirm the issue, which should help us to assess how bad the performance issue is depending on projects.

On the LTS, our patch policy is to fix vulnerabilities, blocker bugs, and big performance issues. Depending on our benchmark, we will assess if the performance issue is big enough to justify a backport to the LTS.

on the other hand, the latest release of SonarQube (9.7) already uses postgresql:42.4.1, which have the tcpNoDelay enabled by default. If upgrading to latest is possible for you, that’s an option you should consider.

1 Like

Our benchmark shows that with tcpNoDelay=false, persisting live measures take about 2x longer. We will discuss this result internally to decide if it justifies backporting the fix to the LTS. Thanks again for reporting this :+1:

1 Like

Affected versions:
8.9.8 (Release: 4 Apr 2022)
8.9.9 (Release: 16 Jun 2022)
8.9.10 (Release: 18 Oct 2022)