Sonarqube no longer serving pages - elastic search throwing exception with "too many files."

sonarqube
elasticsearch
(Alex McLellan) #1

Unfortunately we’ve encountered an issue with our sonarqube installation yesterday, which is a developer trial we’ve had running for a week and a half. It’s no longer serving pages. Looking at the logs for sonarqube, it appears that elastic search has an issue with “Too many open files”, I’ve attached the logs. The setup is a simple 18.04 Ubuntu system, and I followed this installation guide to install it.

Following this issue I found this page in the community forum which told me to look at this page in your documentation regarding requirements. I checked out requirements for linux on the system we’re using;

    sonarqube@Indra-S010:/opt/sonarqube$ sysctl vm.max_map_count

    vm.max_map_count = 65530

    sonarqube@Indra-S010:/opt/sonarqube$ sysctl fs.file-max

    fs.file-max = 3265080

    sonarqube@Indra-S010:/opt/sonarqube$ ulimit -n

    65536

    sonarqube@Indra-S010:/opt/sonarqube$ ulimit -u

    127873

As vm.max_map_count was lower there than what was suggested should be used, I increased it to 262144 by editing /etc/sysctl.conf and adding vm.max_map_count = 262144 to that document and ran sudo sysctl -p to apply the changes straight away, verified that change had taken place;

    sonarqube@Indra-S010:/opt/sonarqube$ sysctl vm.max_map_count
vm.max_map_count = 262144

but unfortunately, it still appears to have the issue with elasticsearch and it’s not serving web pages. Is there anything else I can do to recover this installation?

(G Ann Campbell) #2

Hi,

Recovery is actually pretty easy. A minimum would be to just bounce SonarQube (close all the files & start ES fresh). If you want to be safe that you fully address it the first time, delete $SONARQUBE_HOME/data/es5 before you start the server back up. That will force ES to reindex. I think it’s possible that indexing might have gotten out of whack during your incident.

Going forward, having the correct settings should prevent problems in the future. Please do let us know if it doesn’t.

 
Ann

(Alex McLellan) #3

Hi Ann, I tried bouncing the daemon, without success, a few times. But removing the /data/es6 directory did the trick.

Thank you very much!

1 Like