Sonarqube logs shipping to Logstash using elastic filebeat sidecar container

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension) Sonarqube 9.5

  • what are you trying to achieve
    I’m trying to send log to Logstash using filebeat. I have installed Sonarube enterprise in our AKS cluster using the helm chart. It is working fine. I want to ship the sonarqube log to logstash by sidecar logging method. I run filebeat side container

  • what have you tried so far to achieve this
    I have created filebeat Configmap and a filebeat sidecar container along with Sonarqube container.
    These are my configuration:
    ----------------------------------- values.yaml --------------------------------
    edition: enterprise

    enabled: false

    enable: true
    jdbcUsername: “username”
    jdbcSecretName: “secretname”
    jdbcSecretPasswordKey: “password”
    jdbcUrl: “jdbc:postgresql://servername:5432”

    enabled: true
    ingressClassName: nginx

    • secretName: soanrqube-certificate
    • name:
    • name: filebeat-sonar
      imagePullPolicy: Always
      • name: sonarqube
        mountPath: /opt/sonarqube/logs
        subPath: logs
      • name: filebeat-config
        mountPath: /etc/filebeat.yml
        subPath: filebeat.yml
        readOnly: true
      • name: filebeat-config
        name: filebeat-configmap
        defaultMode: 0600
        apiVersion: v1
        kind: ConfigMap
        name: filebeat-configmap
        filebeat.yml: |-
      • type: container
        enabled: true
        • /opt/sonarqube/logs/*
          hosts: [“logstashhost:5044”]

Both container are running fine but no logs is being send to logstash Kibana portal. When I check the log of filebeat container, it reading the filename":"log/log.go

{“log.level”:“info”,“@timestamp”:“2022-08-29T09:56:32.283Z”,“log.logger”:“monitoring”,“log.origin”:{“”:“log/log.go”,“file.line”:185},“message”:“Non-zero metrics in the last 30s”,“”:“filebeat”,“monitoring”:{“metrics”:{“beat”:{“cgroup”:{“cpuacct”:{“total”:{“ns”:3525864}}},“cpu”:{“system”:{“ticks”:37090},“total”:{“ticks”:62800,“value”:0},“user”:{“ticks”:25710}},“handles”:{“limit”:{“hard”:1048576,“soft”:1048576},“open”:9},“info”:{“ephemeral_id”:“15912e26-403a-4e34-9d96-cf49a4b73c14”,“uptime”:{“ms”:439140064},“version”:“8.3.3”},“memstats”:{“gc_next”:18343656,“memory_alloc”:9595456,“memory_total”:4065261872,“rss”:106061824},“runtime”:{“goroutines”:27}},“filebeat”:{“harvester”:{“open_files”:0,“running”:0}},“libbeat”:{“config”:{“module”:{“running”:0}},“output”:{“events”:{“active”:0}},“pipeline”:{“clients”:0,“events”:{“active”:0}}},“registrar”:{“states”:{“current”:0}},“system”:{“load”:{“1”:0.08,“15”:0.09,“5”:0.11,“norm”:{“1”:0.01,“15”:0.0113,“5”:0.0138}}}},“ecs.version”:“1.6.0”}}

Please suggest.

Hello @Sumit_Singh1 thanks a lot for your post.

From my perspective your configuration looks really fine except for the type: container on the filebeat config.

The container type means that filebeat expect to find logs that are written by the CRI ( mostly docker daemon ) on /var/logs/containers/* on a specific log format ( mostly json )

Regarding your log/log.go, filebeat is written in go so its probably its own log formated in a json format.

I have not tried reproducing the issue but could you try this and get back to us ? :

  • exec into the sidecar container and check if the logs are correctly written there and are readable.
  • if that’s okay, change to type: log

Apart from that what you want to achieve seems a valid use case, and should work without issues.

Best regards,

1 Like

Hi @jeremy.cotineau

Thanks for your time to look into my issue.

Yes, I changed type: filestream and it start sending the logs.

We can mark this post as done.

Best Regards

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.