6.7.5 LTS - Unresponsive ES - Always need to reindex ES

elasticsearch

(Jeannie Chan) #1

Hello,

We are upgrading from SonarQube 5.6 LTS version (Community) to SonarQube 6.7.5 LTS version (build 38563) with Enterprise License.
We follow instructions from here https://docs.sonarqube.org/display/SONARQUBE67/SonarSource+Editions and have upraded our VMs according to the new spec here: https://docs.sonarqube.org/latest/requirements/requirements/. At the large part, it works. However, as usage increase, SonarQube performance went down to irresponsive. The dashboard no longer loads any projects and report uploads came to a halt. We have look into the following area:

  • LOC: We are way below from the LOC of our license
  • Application server Stat: RAM, CPU, Hard disk all at normal state
  • Database (MSSQL 2016): Performance is fine, though CPU usage is higher than previous version. No error in DB logs
  • All logs, include CE, Web, ES, Access, turned on DEBUG level, there is no Error/Warning in any of these logs.
  • IIS logs: no errors
  • We have also ensure that our MSSQL DB has set READ COMMITTED SNAPSHOT as per documentation.

We have tried:

  • Restart the application
  • Restart the server
    And nothing improve the performance of it

We have re-index ES (by renaming the /data/es5) and the performance return to be very snappy. However, after couple hours, the performance went back to irresponsive. We have exhausted our wits to diagnose any possible logs. Any pointer will be greatly appreciated. Thanks!