Sonar with Aurora Serverless

Mike_Klaene · April 27, 2020, 6:06pm

Quick question:
I was thinking of trying out SonarQube, backed with Aurora Postgres Serverless. I was curious as to whether SonarQube’s database interaction is driven solely by the invocation of a scan, or if there were regular/batch database processes that also occurred.

Aurora Serverless might not work well in a scenario where the server is continually checking in with the database (versus only when a scan is invoked, in which case it might work very well).

Thanks.

Antoine · April 28, 2020, 6:06am

Hello @Mike_Klaene,

SonarQube is a whole web application, for which persisting analysis data is just one task out of many. So they are indeed a lot of DB interactions, a lot more than just when a scan occurs (user login, issue status change, a lot of UI info SELECTed from the DB, etc. in fact it’s not possible to make an exhaustive list).

I don’t know much about Aurora Postgres Serverless but from my understanding, it’s ultimately based on a PostgreSQL DB, with some fancy up/down-scale mechanisms on top of that. I guess from an application point of view it’s just a PostgreSQL DB, so I don’t see why it won’t work as usual.

Why do you foresee a difference depending on the interactions volume DB <> Application?

Mike_Klaene · April 28, 2020, 5:05pm

Thanks for the feedback.
This is what I assumed - that there are quite a number of instances where the application needs to communicate to the database.

The reason I asked is I was trying to get a feel for whether, for example, there were hourly background jobs that ran to monitor, grab stats, etc. or whether it’s mostly limited to direct user actions.

For Aurora Serverless, generally speaking, the costs are significantly lower when db usage is infrequent and/or predictable. For continuous database interaction, it can actually prove to be more costly than the traditional database approach.

I think I’ll just have to experiment a bit and see what works best, but your guidance gives me something to work from.

Thanks.

Antoine · April 28, 2020, 5:33pm

Ah okok, I didn’t think about the costs aspect, I thought it was more a technical related question.

There are not a lot of asynchronous background jobs (hourly checks, report aggregation, etc.), meaning that if there is no user logged-in nor code analysis ongoing, the number of queries sent to the DB is pretty low, even null.
BUT, I’m also quite sure that the JDBC Driver for PostgreSQL has a mechanism to maintain a pool open, or at least to regularly check the connectivity, so it does not only depends on SQ.

All in all, you should monitor DB activity directly to be sure about it. You can also check SQ logs (web.log and ce.log) in TRACE mode to have a rough idea.

Mike_Klaene · April 28, 2020, 6:32pm

Ok. Thanks. Forgot about connection pool.
This would be a very small installation initially, for a single team. Wondering if I could set
sonar.jdbc.minIdle=0 to get around this…

github.com

SonarSource/sonarqube/blob/master/sonar-application/src/main/assembly/conf/sonar.properties#L68


# enabled with default sizes (50, see property sonar.web.http.maxThreads)
# then sonar.jdbc.maxActive should be 1.2 * 50 = 60.
#sonar.jdbc.maxActive=60


# The maximum number of connections that can remain idle in the
# pool, without extra ones being released, or negative for no limit.
#sonar.jdbc.maxIdle=5


# The minimum number of connections that can remain idle in the pool,
# without extra ones being created, or zero to create none.
#sonar.jdbc.minIdle=2


# The maximum number of milliseconds that the pool will wait (when there
# are no available connections) for a connection to be returned before
# throwing an exception, or <= 0 to wait indefinitely.
#sonar.jdbc.maxWait=5000


#sonar.jdbc.minEvictableIdleTimeMillis=600000
#sonar.jdbc.timeBetweenEvictionRunsMillis=30000

Karthik_N_S · December 23, 2020, 2:18am

@Mike_Klaene, Were you successful in getting this connection established. I am trying to use Aurora Serverless, but Sonar doesn’t seem to connect to aurora server less. I can confirm that the ec2 running sonar has connectivity to aurora ads and able to psql into the db

Thanks
K

Karthik_N_S · December 23, 2020, 4:01am

Nevermind, it was a jdbc driver version issue. Sonar was unable to connect Aurora Serverless with JDBC driver supplied by sonar “8.5.0.37579” which is 42.2.16. I had to downgrade to 42.2.14. https://github.com/quarkusio/quarkus/issues/12022