Hello,
I’m trying to deploy sonarqube in ecs with fargate, and with rds postgress, both in separate vpc.
The problem is that when deploying all the components in aws sonarqube always starts with the local database, I have checked the peering between both vpc and it is correct.
Do you know if what I’m trying to do is valid or if the connection is not possible?
Regards;
Hey there.
I don’t know a lot about deploying with ECS/Fargate, but regardless of your deployment style, you’ll need to connect SonarQube to the database by specifying sonar.jdbc.* configuration either in the conf/sonar.properties file or via Environment variables when deploying with Docker.
Well I have implemented something similar to docker:
AWSTemplateFormatVersion: '2010-09-09'
Description: 'Task Definition for SonarQube on ECS Fargate with Secrets Manager'
Resources:
# IAM Role for ECS Task
ECSTaskExecutionRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action: 'sts:AssumeRole'
Policies:
- PolicyName: ECSFargateTaskExecutionPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- efs:ClientMount
- efs:ClientWrite
Resource: '*'
- Effect: Allow
Action:
- ecr:GetDownloadUrlForLayer
- ecr:BatchGetImage
- ecr:GetAuthorizationToken
Resource: '*'
- Effect: Allow
Action:
- secretsmanager:GetSecretValue
Resource: '*'
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:CreateLogStream
- logs:PutLogEvents
- logs:DescribeLogStreams
Resource: '*'
# IAM Role for the Task itself
ECSTaskRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action: 'sts:AssumeRole'
Policies:
- PolicyName: ECSFargateTaskRolePolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- ecs:StartTask
- ecs:StopTask
- ecs:RunTask
- efs:*
Resource: '*'
LogGroupSonar:
Type: 'AWS::Logs::LogGroup'
Properties:
LogGroupName: '/ecs/'
RetentionInDays: 7
# ECS Task Definition
ECSTaskDefinition:
Type: 'AWS::ECS::TaskDefinition'
Properties:
Family: 'SonarQubeTask'
Cpu: '2048'
Memory: '8192'
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
ExecutionRoleArn: !GetAtt ECSTaskExecutionRole.Arn
TaskRoleArn: !GetAtt ECSTaskRole.Arn
RuntimePlatform:
OperatingSystemFamily: LINUX
CpuArchitecture: X86_64
ContainerDefinitions:
- Name: sonarqube
Image: "{{resolve:ssm:/config/test/ecs/sonarqube-image}}"
Essential: true
PortMappings:
- ContainerPort: 9000
HostPort: 9000
Protocol: tcp
Name: sonar-port
AppProtocol: http
Environment:
- Name: SONARQUBE_JDBC_USERNAME
Value: sonaradmin
- Name: SONARQUBE_JDBC_URL
Value: "{{resolve:ssm:/config/rds/db-endpoint}}"
- Name: SET_CONTAINER_TIMEZONE
Value: 'true'
- Name: TZ
Value: 'America/Colombia'
Secrets:
- Name: SONARQUBE_JDBC_PASSWORD
ValueFrom: 'arn:aws:secretsmanager:us-west-test:password::'
MountPoints:
- SourceVolume: sonarqube_data
ContainerPath: /opt/sonarqube/data
ReadOnly: false
- SourceVolume: sonarqube_logs
ContainerPath: /opt/sonarqube/logs
ReadOnly: false
- SourceVolume: sonarqube_extensions
ContainerPath: /opt/sonarqube/extensions
ReadOnly: false
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref LogGroupSonar
awslogs-stream-prefix: 'sonarqube'
mode: non-blocking
max-buffer-size: 25m
awslogs-region: !Ref 'AWS::Region'
Volumes:
- Name: sonarqube_data
EfsVolumeConfiguration:
FileSystemId: !ImportValue 'SonarQubeEFSStackEFSFileSystemId'
AuthorizationConfig:
AccessPointId: !ImportValue 'EFSAccessPointDataId'
IAM: 'DISABLED'
TransitEncryption: 'ENABLED'
RootDirectory: '/'
- Name: sonarqube_logs
EfsVolumeConfiguration:
FileSystemId: !ImportValue 'SonarQubeEFSStackEFSFileSystemId'
AuthorizationConfig:
AccessPointId: !ImportValue 'EFSAccessPointLogsId'
IAM: 'DISABLED'
TransitEncryption: 'ENABLED'
RootDirectory: '/'
- Name: sonarqube_extensions
EfsVolumeConfiguration:
FileSystemId: !ImportValue 'SonarQubeEFSStackEFSFileSystemId'
AuthorizationConfig:
AccessPointId: !ImportValue 'EFSAccessPointExtensionsId'
IAM: 'DISABLED'
TransitEncryption: 'ENABLED'
RootDirectory: '/'
But when starting the service it always starts with the local database, I also did the test using other environment variables for example:
SONAR_JDBC_USERNAME
SONAR_JDBC_URL
SONAR_JDBC_URL
Do you have any comments?
Well, you definitely need to make sure you use SONAR_JDBC_USERNAME,
SONAR_JDBC_PASSWORD and SONAR_JDBC_URL instead of SONARQUBE_JDBC_*. Are you sure you’ve changed all 3?
Yes, I’m sure that when I deploy with those variables:
SONAR_JDBC_USERNAME
SONAR_JDBC_URL
SONAR_JDBC_URL
The service never finishes deploying, and I see errors related to elastic search. However, I don’t see any database connection errors.
What are the errors?
These are the errors I see from: cloud watch:
2024.11.26 16:14:22 INFO app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
2024.11.26 16:14:22 INFO app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:41363]
2024.11.26 16:14:22 INFO app[][o.s.a.ProcessLauncherImpl] Launch process[ELASTICSEARCH] from [/opt/sonarqube/elasticsearch]: /opt/java/openjdk/bin/java -Xms4m -Xmx64m -XX:+UseSerialGC -Dcli.name=server -Dcli.script=./bin/elasticsearch -Dcli.libs=lib/tools/server-cli -Des.path.home=/opt/sonarqube/elasticsearch -Des.path.conf=/opt/sonarqube/temp/conf/es -Des.distribution.type=tar -cp /opt/sonarqube/elasticsearch/lib/*:/opt/sonarqube/elasticsearch/lib/cli-launcher/* org.elasticsearch.launcher.CliToolLauncher
2024.11.26 16:14:22 INFO app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
2024.11.26 16:14:29 INFO es[][o.e.n.NativeAccess] Using [jna] native provider and native methods for [Linux]
2024.11.26 16:14:30 INFO es[][o.e.n.Node] version[8.13.4], pid[88], build[tar/da95df118650b55a500dcc181889ac35c6d8da7c/2024-05-06T22:04:45.107454559Z], OS[Linux/5.10.228-219.884.amzn2.x86_64/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/17.0.12/17.0.12+7]
2024.11.26 16:14:30 INFO es[][o.e.n.Node] JVM home [/opt/java/openjdk], using bundled JDK [false]
2024.11.26 16:14:30 INFO es[][o.e.n.Node] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -XX:ReplayDataFile=logs/replay_pid%p.log, -Des.distribution.type=tar, -XX:+UseG1GC, -Djava.io.tmpdir=/opt/sonarqube/temp, -XX:ErrorFile=/opt/sonarqube/logs/es_hs_err_pid%p.log, -Xlog:disable, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djna.tmpdir=/opt/sonarqube/temp, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=COMPAT, -Des.enforce.bootstrap.checks=true, -Xmx2G, -Xms2G, -XX:+HeapDumpOnOutOfMemoryError, -XX:MaxDirectMemorySize=1073741824, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, --module-path=/opt/sonarqube/elasticsearch/lib, --add-modules=jdk.net, --add-modules=ALL-MODULE-PATH, -Djdk.module.main=org.elasticsearch.server]
2024.11.26 16:14:30 INFO es[][o.e.n.Node] Default Locale [en_US]
2024.11.26 16:14:32 INFO es[][o.e.p.PluginsService] loaded module [rest-root]
Regards;
No errors to be seen, but if it’s stalling I would suggest raising the log level to SONAR_LOG_LEVEL=DEBUG to see if there’s more information about what’s happening.
That would be at the compose/task definition level?
Yes, the same place you’re setting these.
HI, I have enabled the debug parameter and these are the logs that are shown, I also enabled traffic for all vpc and still the service deployment never ends.
1733264372664,java.net.ConnectException: Connection refused
1733264372664, at java.base/sun.nio.ch.Net.pollConnect(Native Method)
1733264372664, at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source)
1733264372664, at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
1733264372664, at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)
1733264372664, at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)
1733264372664, at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
1733264372664, at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
1733264372664, at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
1733264372664, at java.base/java.lang.Thread.run(Unknown Source)
1733264372664,2024.12.03 17:19:32 DEBUG app[][o.a.h.i.n.c.InternalHttpAsyncClient] [exchange: 142] connection request failed
1733264372664,2024.12.03 17:19:32 DEBUG app[][o.e.c.RestClient] request [GET http://localhost:9001/] failed
1733264372664,java.net.ConnectException: Connection refused
1733264372664, at java.base/sun.nio.ch.Net.pollConnect(Native Method)
1733264372664, at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source)
1733264372664, at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
1733264372664, at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)
1733264372664, at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)
1733264372664, at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
1733264372664, at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
1733264372664, at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
1733264372664, at java.base/java.lang.Thread.run(Unknown Source)
1733264372664,2024.12.03 17:19:32 DEBUG app[][o.e.c.RestClient] updated [[host=http://localhost:9001]] already in blacklist
1733264372765,2024.12.03 17:19:32 DEBUG app[][o.a.h.i.n.c.MainClientExec] [exchange: 143] start execution
1733264372765,2024.12.03 17:19:32 DEBUG app[][o.a.h.c.p.RequestAddCookies] CookieSpec selected: default
1733264372766,2024.12.03 17:19:32 DEBUG app[][o.a.h.c.p.RequestAuthCache] Re-using cached 'basic' auth scheme for http://localhost:9001
1733264372766,2024.12.03 17:19:32 DEBUG app[][o.a.h.c.p.RequestAuthCache] No credentials for preemptive authentication
1733264372766,2024.12.03 17:19:32 DEBUG app[][o.a.h.i.n.c.InternalHttpAsyncClient] [exchange: 143] Request connection for {}->http://localhost:9001
1733264372766,2024.12.03 17:19:32 DEBUG app[][o.a.h.i.n.c.PoolingNHttpClientConnectionManager] Connection request: [route: {}->http://localhost:9001][total kept alive: 0; route allocated: 0 of 10; total allocated: 0 of 30]
1733264372767,2024.12.03 17:19:32 DEBUG app[][o.a.h.i.n.c.PoolingNHttpClientConnectionManager] Connection request failed
1733264372767,java.net.ConnectException: Connection refused
1733264372767, at java.base/sun.nio.ch.Net.pollConnect(Native Method)
1733264372767, at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source)
1733264372767, at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
1733264372767, at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)
1733264372767, at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)
1733264372767, at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
1733264372767, at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
1733264372767, at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
1733264372767, at java.base/java.lang.Thread.run(Unknown Source)
1733264372767,2024.12.03 17:19:32 DEBUG app[][o.a.h.i.n.c.InternalHttpAsyncClient] [exchange: 143] connection request failed
1733264372767,2024.12.03 17:19:32 DEBUG app[][o.e.c.RestClient] request [GET http://localhost:9001/] failed
1733264372767,java.net.ConnectException: Connection refused
1733264372767, at java.base/sun.nio.ch.Net.pollConnect(Native Method)
1733264372767, at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source)
1733264372767, at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
Regards;