# Deployment with Helm Chart fails if database updgrade is needed

**URL:** <https://community.sonarsource.com/t/deployment-with-helm-chart-fails-if-database-updgrade-is-needed/55521>\
**Category:** SonarQube Server / Community Build\
**Tags:** 8-9-lts-upgrade, kubernetes, web, upgrade, helm\
**Created:** [December 23, 2021, 2:43pm UTC](https://community.sonarsource.com/t/deployment-with-helm-chart-fails-if-database-updgrade-is-needed/55521 "2021-12-23T14:43:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![apa64](https://dub1.discourse-cdn.com/sonarsource/user_avatar/community.sonarsource.com/apa64/32/28728_2.png) [@apa64](https://community.sonarsource.com/u/apa64)\
**Post date:** [December 23, 2021, 2:43pm UTC](https://community.sonarsource.com/t/deployment-with-helm-chart-fails-if-database-updgrade-is-needed/55521/1 "2021-12-23T14:43:27Z")

</div>

# Helm Chart has invalid livenessProbe when database upgrade is needed

- versions used (SonarQube, Scanner, Plugin, and any relevant extension)
  - **Chart** : `sonarqube/sonarqube-lts`, exact version: sonarqube-lts-1.0.23+179
  - **SonarQube application** : 8.9.6 LTS
  - **old SonarQube application** : 8.5.1-community

- error observed (wrap logs/code around triple quote ``` for proper formatting)
  - Application pod ends up in crash loop.

## Problem

- SonarQube won’t start completely up so the Kubernetes probe fails and Kubernetes kills the process.

## Reason

- SQ’s “web” process is in “safe mode” and reports status `RED` to `livenessProbe`.
- “web” is in “safe mode” because the database needs upgrade for the new SQ version. This is quite normal, you do it by going to SQ\_URL/setup and let it upgrade. But web stays in safe mode until upgrade is complete.
- Kubernetes does not mark the pod as “Running” and “Healthy” because `livenessProbe` fails. Pod is soon restarted and the loop goes on.

## Workaround

- The `livenessProbe` is defined in `templates/sonarqube-sts.yaml`. On line 302 it accepts `health: GREEN` or `health: YELLOW` responses. Modify this to accept also `health: RED`.
- Install:

```shell
# The folder sonarqube-lts-1.0.23+179 contains the modified chart.
$ helm3 -n sonarqube install sonarqube sonarqube-lts-1.0.23+179 -f values.yaml

```

- Now the probe passes and Kubernetes lets traffic go to the pod. Browse to `../setup`, upgrade the database, redeploy with unmodified `sonarqube/sonarqube-lts` chart.

---

<div class="post-metadata">

**Author:** ![Sylvain\_Combe](https://dub1.discourse-cdn.com/sonarsource/user_avatar/community.sonarsource.com/sylvain_combe/32/3972_2.png) [@Sylvain\_Combe](https://community.sonarsource.com/u/Sylvain_Combe)\
**Post date:** [December 28, 2021, 11:14am UTC](https://community.sonarsource.com/t/deployment-with-helm-chart-fails-if-database-updgrade-is-needed/55521/2 "2021-12-28T11:14:31Z")

</div>

Hello @apa64  
thanks for your feedback, and for sharing a work around! 👍

As you’ve seen with the [LTS helm chart](https://github.com/SonarSource/helm-chart-sonarqube/blob/cb6cb70c12ce19ba6bf1f947e9d74d2d9d77c845/charts/sonarqube-lts/templates/sonarqube-sts.yaml#L302) definition, the pod Liveness is directly mapped to the `api/system/health` SonarQube API endpoint, which provides a slightly different information:  
 ![image](https://europe1.discourse-cdn.com/sonarsource/uploads/sonarcommunity/original/3X/0/3/03bf24c4ec241bfef8868931fd366819369f3ad1.png)

That’s the case only for the LTS chart though. With [SONAR-15239](https://jira.sonarsource.com/browse/SONAR-15239) a liveness endpoint was added to SonarQube 9.1+ API. The 9.x [helmchart relies on this new endpoint](https://github.com/SonarSource/helm-chart-sonarqube/blob/cb6cb70c12ce19ba6bf1f947e9d74d2d9d77c845/charts/sonarqube/templates/sonarqube-sts.yaml#L308) and should not suffer from any such upgrade problem.

**Warning** : the last step of the work around described by Antii is to come back to the original chart, this is important as the work around practically disables the `livenessProbe`

**Alternative** : Another work around would be to apply the upgrade before the move to K8S.

---

<div class="post-metadata">

**Author:** ![system](https://dub1.discourse-cdn.com/sonarsource/user_avatar/community.sonarsource.com/system/32/31083_2.png) [@system](https://community.sonarsource.com/u/system)\
**Post date:** [January 4, 2022, 6:10pm UTC](https://community.sonarsource.com/t/deployment-with-helm-chart-fails-if-database-updgrade-is-needed/55521/3 "2022-01-04T18:10:34Z")

</div>

Closed on January 4, 2022, 6:10pm UTC

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.