After upgrading from SonarQube 8.9.9 to 9.9.2 we randomly see kerberos authention issue

SonarQube Server / Community Build 9.9 LTA Upgrade
1

After upgrading from SonarQube 8.9.9 Community to 9.9.2, server starts up fine but after hours of running, it seems it disconnects from the MSSQL server:

2023.09.28 11:54:42 ERROR web[][o.s.s.p.w.RootFilter] Processing of request /api/measures/component?additionalFields=period%2Cmetrics&metricKeys=alert_status%2Cquality_gate_details&componen
t=com.flowtraders.fi.midmodels%3Afi-mid-models%3Afeature-OAS-5821 failed
org.apache.ibatis.exceptions.PersistenceException:
### Error querying database.  Cause: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 8000ms.
### The error may exist in org.sonar.db.user.UserTokenMapper
### The error may involve org.sonar.db.user.UserTokenMapper.selectByTokenHash
### The error occurred while executing a query
### Cause: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 8000ms.
        at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)

Our kerberos ticket is valid:

root@<SONAR.HOSTNAME>:~# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: svc-nlsqlsonar-java@OUR.REDACTED.REALM

Valid starting       Expires              Service principal
09/28/2023 08:17:01  09/28/2023 18:17:01  krbtgt/OUR.REDACTED.REALM@OUR.REDACTED.REALM
        renew until 10/02/2023 16:15:32

And simply shutting down SonarQube and starting it back up (without renewing the kerberos ticket) makes it work again, until it randomly happens again after a few hours. Servers are in the same network, has the same NTP servers etc.

We are running SonarQube on Debian Bullseye11.5, which is using OpenJDK Runtime Environment (build 17.0.6+10-LTS) against a MSSQL 2016 server.

Our jdbc connector string looks like this:
sonar.jdbc.url=jdbc:sqlserver://<REDACTED_SERVER_NAME>;databaseName=SonarJava;integratedSecurity=true;trustServerCertificate=true;authenticationScheme=JavaKerberos

I verified that we use Latin1_General_CS_AS

Can possibly be related to these issues, CORRECTION: we still see these errors.

2

Hi,

You’re hitting this:

SONAR-19513 HikariCP get exhausted on MSSQL causing SQ instance to freeze

I’m sorry, but I don’t currently have a workaround for you. The team has been working on it, however, and I expect it to be part of the eventual 9.9.3 release.

 
:frowning:
Ann

3

Is there anything we should avoid in our database config that might allow us to get it working more stable? I assume this can’t affect every 9.9.2 LTS users connecting to MSSQL 2017, so we must be doing something special?

Kind regards
Jonas

4

Hi Jonas,

In fact, after answering, I poked around a little more. We don’t actually expect there to be a code fix for this. The Jira ticket I pointed you to isn’t updated because we’re waiting for final confirmation, but it looks like settings will probably handle this. Try setting fixed connection pool and indefinite lifetime for the connections when using Kerberos and SQL Server:

sonar.jdbc.minIdle=25
sonar.jdbc.maxActive=25
sonar.jdbc.maxLifetime=0
sonar.jdbc.maxWait=30000

And since this isn’t finalized yet, we’d love to have feedback, either way.

 
Thx,
Ann

1 Like
5

Hi Ann,

We are testing these settings now, and hope it works, or we will be forced to rollback the upgrade as we can’t be babysitting it until a fix is found. I will keep you updated if this solved our issue.

Kind regards,

Jonas

2 Likes
6

thanks, these settings have worked for us! thanks a ton.

3 Likes