DB migration issue - During sonar upgrade from 6.7-RC1 to 7.9.6

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension) current version - 6.7-RC1
  • what are you trying to achieve Trying to upgrade from current version to 7.9.6
  • what have you tried so far to achieve this.
    1. download the sonarqube - 7.9.6
    2. Stop the previous version of sonar
    3. changed the JDBC url and credentails
    4. start the sonar

Facing the below error:

2021.08.03 19:05:13 INFO  web[][DbMigrations] #1952 'Create builtin quality gate if required'...
2021.08.03 19:05:13 ERROR web[][DbMigrations] #1952 'Create builtin quality gate if required': failure | time=29ms
2021.08.03 19:05:13 ERROR web[][DbMigrations] Executed DB migrations: failure | time=30ms
2021.08.03 19:05:13 ERROR web[][o.s.s.p.d.m.DatabaseMigrationImpl] DB migration failed | time=116ms
2021.08.03 19:05:13 ERROR web[][o.s.s.p.d.m.DatabaseMigrationImpl] DB migration ended with an exception
org.sonar.server.platform.db.migration.step.MigrationStepExecutionException: Execution of migration step #1952 'Create builtin quality gate if required' failed
        at org.sonar.server.platform.db.migration.step.MigrationStepsExecutorImpl.execute(MigrationStepsExecutorImpl.java:79)
        at org.sonar.server.platform.db.migration.step.MigrationStepsExecutorImpl.execute(MigrationStepsExecutorImpl.java:67)
        at java.base/java.lang.Iterable.forEach(Iterable.java:75)
        at org.sonar.server.platform.db.migration.step.MigrationStepsExecutorImpl.execute(MigrationStepsExecutorImpl.java:52)
        at org.sonar.server.platform.db.migration.engine.MigrationEngineImpl.execute(MigrationEngineImpl.java:68)
        at org.sonar.server.platform.db.migration.DatabaseMigrationImpl.doUpgradeDb(DatabaseMigrationImpl.java:105)
        at org.sonar.server.platform.db.migration.DatabaseMigrationImpl.doDatabaseMigration(DatabaseMigrationImpl.java:80)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "id" violates not-null constraint  Detail: Failing row contains (null, Sonar way, 2021-08-03 19:05:13.614, null, t, AXsNZ9SPbY_QBFmueOvX).
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
        at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
        at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
        at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
        at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:132)
        at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:94)
        at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:94)
        at org.sonar.server.platform.db.migration.step.UpsertImpl.execute(UpsertImpl.java:68)
        at org.sonar.server.platform.db.migration.version.v70.CreateBuiltInQualityGate.execute(CreateBuiltInQualityGate.java:58)
        at org.sonar.server.platform.db.migration.step.DataChange.execute(DataChange.java:45)
        at org.sonar.server.platform.db.migration.step.MigrationStepsExecutorImpl.execute(MigrationStepsExecutorImpl.java:75)
        ... 9 common frames omitted

Hi Team,

 Can anyone kindly help on the above issues.

Regards,
Sangeetha K

Hi Sangeetha,

You’re coming from a very old version which was never actually a production version. You should probably restore your DB backup and upgrade first to 6.7.7 - yes upgrades within versions aren’t usually required, but there’s a reason it was a release candidate - and then try the upgrade to 7.9.6.

 
:woman_shrugging:
Ann

Hi @ganncamp Ann,

Thanks for the update. I’ve successfully upgraded to 6.7.7. Facing the below issue after upgrading from 6.7.7 to 7.9.6.

2021.08.04 18:46:59 INFO web[o.s.s.p.d.m.c.PostgresCharsetHandler] Verify that database charset supports UTF8
2021.08.04 18:47:00 INFO web[o.s.s.p.Platform] Database needs to be migrated. Please refer to Upgrade the Server | SonarQube Docs
2021.08.04 18:47:00 INFO web[o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceFilter@38bdaf69 [pattern=UrlPattern{inclusions=[/api/system/migrate_db., …], exclusions=[/api/properties, …]}]
2021.08.04 18:47:00 INFO web[o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
2021.08.04 19:00:22 INFO web[o.s.s.p.d.m.DatabaseMigrationImpl] Starting DB Migration and container restart
2021.08.04 19:00:22 INFO web[DbMigrations] Executing DB migrations…
2021.08.04 19:00:22 INFO web[DbMigrations] #1930 ‘Add QUALITY_GATES.IS_BUILT_IN’…
2021.08.04 19:00:22 INFO web[DbMigrations] #1930 ‘Add QUALITY_GATES.IS_BUILT_IN’: success | time=14ms
2021.08.04 19:00:22 INFO web[DbMigrations] #1931 ‘Populate QUALITY_GATES.IS_BUILT_IN’…
2021.08.04 19:00:22 INFO web[o.s.s.p.d.m.s.MassUpdate] 0 quality_gates processed (0 items/sec)
2021.08.04 19:00:22 INFO web[DbMigrations] #1931 ‘Populate QUALITY_GATES.IS_BUILT_IN’: success | time=21ms
2021.08.04 19:00:22 INFO web[DbMigrations] #1932 ‘Make QUALITY_GATES.IS_BUILT_IN not null’…
2021.08.04 19:00:22 INFO web[DbMigrations] #1932 ‘Make QUALITY_GATES.IS_BUILT_IN not null’: success | time=7ms
2021.08.04 19:00:22 INFO web[DbMigrations] #1933 ‘Remove quality gates loaded templates’…
2021.08.04 19:00:22 INFO web[o.s.s.p.d.m.s.MassUpdate] 1 delete loaded templates for quality gate processed (0 items/sec)
2021.08.04 19:00:22 INFO web[DbMigrations] #1933 ‘Remove quality gates loaded templates’: success | time=42ms
2021.08.04 19:00:22 INFO web[DbMigrations] #1934 ‘Rename quality gate “SonarQube way” to “Sonar way”’…
2021.08.04 19:00:22 INFO web[o.s.s.p.d.m.s.MassUpdate] 0 quality gates processed (0 items/sec)
2021.08.04 19:00:22 INFO web[DbMigrations] #1934 ‘Rename quality gate “SonarQube way” to “Sonar way”’: success | time=8ms
2021.08.04 19:00:22 INFO web[DbMigrations] #1935 ‘Drop LOADED_TEMPLATES table’…
2021.08.04 19:00:22 INFO web[DbMigrations] #1935 ‘Drop LOADED_TEMPLATES table’: success | time=21ms
2021.08.04 19:00:22 INFO web[DbMigrations] #1936 ‘Delete person measures’…
2021.08.04 19:00:57 INFO web[o.s.s.p.d.m.s.MassUpdate] 19592 snapshots processed (326 items/sec)
2021.08.04 19:00:57 INFO web[DbMigrations] #1936 ‘Delete person measures’: success | time=34755ms
2021.08.04 19:00:57 INFO web[DbMigrations] #1937 ‘Drop index on project_measures.person_id’…
2021.08.04 19:00:57 INFO web[DbMigrations] #1937 ‘Drop index on project_measures.person_id’: success | time=19ms
2021.08.04 19:00:57 INFO web[DbMigrations] #1938 ‘Create table live_measures’…
2021.08.04 19:00:57 ERROR web[DbMigrations] #1938 ‘Create table live_measures’: failure | time=67ms
2021.08.04 19:00:57 ERROR web[DbMigrations] Executed DB migrations: failure | time=34962ms
2021.08.04 19:00:57 ERROR web[o.s.s.p.d.m.DatabaseMigrationImpl] DB migration failed | time=35117ms
2021.08.04 19:00:57 ERROR web[o.s.s.p.d.m.DatabaseMigrationImpl] DB migration ended with an exception
org.sonar.server.platform.db.migration.step.MigrationStepExecutionException: Execution of migration step #1938 ‘Create table live_measures’ failed
at org.sonar.server.platform.db.migration.step.MigrationStepsExecutorImpl.execute(MigrationStepsExecutorImpl.java:79)
at org.sonar.server.platform.db.migration.step.MigrationStepsExecutorImpl.execute(MigrationStepsExecutorImpl.java:67)
at java.base/java.lang.Iterable.forEach(Iterable.java:75)
at org.sonar.server.platform.db.migration.step.MigrationStepsExecutorImpl.execute(MigrationStepsExecutorImpl.java:52)
at org.sonar.server.platform.db.migration.engine.MigrationEngineImpl.execute(MigrationEngineImpl.java:68)
at org.sonar.server.platform.db.migration.DatabaseMigrationImpl.doUpgradeDb(DatabaseMigrationImpl.java:105)
at org.sonar.server.platform.db.migration.DatabaseMigrationImpl.doDatabaseMigration(DatabaseMigrationImpl.java:80)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalStateException: Fail to execute CREATE TABLE live_measures (uuid VARCHAR (40) NOT NULL,project_uuid VARCHAR (50) NOT NULL,component_uuid VARCHAR (50) NOT NULL,metric_id INTEGER NOT NULL,value NUMERIC (38,20) NULL,text_value VARCHAR (4000) NULL,variation NUMERIC (38,20) NULL,measure_data BYTEA NULL,update_marker VARCHAR (40) NULL,created_at BIGINT NOT NULL,updated_at BIGINT NOT NULL, CONSTRAINT pk_live_measures PRIMARY KEY (uuid))
at org.sonar.server.platform.db.migration.step.DdlChange$Context.execute(DdlChange.java:97)
at org.sonar.server.platform.db.migration.step.DdlChange$Context.execute(DdlChange.java:77)
at org.sonar.server.platform.db.migration.step.DdlChange$Context.execute(DdlChange.java:117)
at org.sonar.server.platform.db.migration.version.v70.CreateTableLiveMeasures.execute(CreateTableLiveMeasures.java:53)
at org.sonar.server.platform.db.migration.step.DdlChange.execute(DdlChange.java:45)
at org.sonar.server.platform.db.migration.step.MigrationStepsExecutorImpl.execute(MigrationStepsExecutorImpl.java:75)
… 9 common frames omitted
Caused by: org.postgresql.util.PSQLException: ERROR: relation “live_measures” already exists
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:266)
at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:175)
at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:175)
at org.sonar.server.platform.db.migration.step.DdlChange$Context.execute(DdlChange.java:82)
… 14 common frames omitted

Regards,
Sangeetha K

Hi Sangeetha,

I’ve flagged this for expert attention but I’m not sure what kind of traction we’ll get on this since you’re starting from such an old version. What’s your investment in this instance? Is it possible it would be easier to just start fresh on 8.9.2 or 9.0.1?

 
Ann

Hi @ganncamp,

Above issue got fixed, Now I’m facing below error while doing DB migration in 7.9.6 .

2021.08.04 20:53:32 INFO  web[][o.s.s.s.RegisterMetrics] Disable metric Configuration File Complexity [mule_configuration_complexity_rating]
2021.08.04 20:53:32 INFO  web[][o.s.s.r.RegisterRules] Register rules
2021.08.04 20:53:34 ERROR web[][o.s.s.p.Platform] Background initialization failed. Stopping SonarQube
org.apache.ibatis.exceptions.PersistenceException:
### Error querying database.  Cause: org.postgresql.util.PSQLException: ERROR: column drk.rule_id does not exist
  Hint: Perhaps you meant to reference the column "drk.rule_uuid".
  Position: 345
### The error may exist in org.sonar.db.rule.RuleMapper
### The error may involve defaultParameterMap
### The error occurred while setting parameters
### SQL: SELECT       drk.uuid,       drk.rule_id as "ruleId",       drk.old_repository_key as "oldRepositoryKey",       drk.old_rule_key as "oldRuleKey",       r.plugin_rule_key as "newRuleKey",       r.plugin_name as "newRepositoryKey",       drk.created_at as "createdAt"     FROM       deprecated_rule_keys drk     LEFT OUTER JOIN rules r on r.id = drk.rule_id
### Cause: org.postgresql.util.PSQLException: ERROR: column drk.rule_id does not exist
  Hint: Perhaps you meant to reference the column "drk.rule_uuid".
  Position: 345
        at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
        at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:149)
        at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:140)
        at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:147)
        at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:80)
        at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:58)
        at com.sun.proxy.$Proxy40.selectAllDeprecatedRuleKeys(Unknown Source)
        at org.sonar.db.rule.RuleDao.selectAllDeprecatedRuleKeys(RuleDao.java:254)

Regards,
Sangeetha K

Hi Team,

Can anyone kindly help us on the above issue.

Regards,
Sangeetha K

Hi Sangeetha,

There’s no one who’s full-time job it is to help people in the community. People help out in the community as they have time and when they have time. You can’t expect immediate turnaround on your questions.

And, since you solved your first problem, it might help others in the future if you followed up here with how you fixed it.

Regarding your new error… that’s a new issue and needs a new thread.

 
:woman_shrugging:
Ann