Microsoft SQLServer DB migration fails when upgrading SonarQube DE from 9.9.0 LTS to 10.4.1

ThE · April 11, 2024, 11:48am

Hi,

I’m facing a DB migration error when upgrading SonarQube Developer Edition from 9.9.0.65466 to 10.4.1.88267.
As far as i understand the upgrade path examples I should be able to directly upgade from 9.9 LTS to > 10.1+?

2024.04.11 13:24:12 INFO  web[][DbMigrations] Executing DB migrations...
2024.04.11 13:24:12 INFO  web[][DbMigrations] #100000 'Remove orphan rules in Quality Profiles'...
2024.04.11 13:24:13 INFO  web[][o.s.s.p.d.m.s.MassUpdate] 0 rows processed (0 items/sec)
2024.04.11 13:24:13 INFO  web[][DbMigrations] #100000 'Remove orphan rules in Quality Profiles': success | time=187ms
2024.04.11 13:24:13 INFO  web[][DbMigrations] #100001 'Drop index 'projects_module_uuid' in the 'Components' table'...
2024.04.11 13:24:13 INFO  web[][DbMigrations] #100001 'Drop index 'projects_module_uuid' in the 'Components' table': success | time=125ms
2024.04.11 13:24:13 INFO  web[][DbMigrations] #100002 'Drop column 'module_uuid' in the 'Components' table'...
2024.04.11 13:24:13 ERROR web[][DbMigrations] #100002 'Drop column 'module_uuid' in the 'Components' table': failure | time=141ms
2024.04.11 13:24:13 ERROR web[][DbMigrations] Executed DB migrations: failure | time=453ms
2024.04.11 13:24:13 ERROR web[][o.s.s.p.d.m.DatabaseMigrationImpl] DB migration failed | time=485ms
2024.04.11 13:24:13 ERROR web[][o.s.s.p.d.m.DatabaseMigrationImpl] DB migration ended with an exception
org.sonar.server.platform.db.migration.step.MigrationStepExecutionException: Execution of migration step #100002 'Drop column 'module_uuid' in the 'Components' table' failed
	at org.sonar.server.platform.db.migration.step.MigrationStepsExecutorImpl.execute(MigrationStepsExecutorImpl.java:79)
	at org.sonar.server.platform.db.migration.step.MigrationStepsExecutorImpl.execute(MigrationStepsExecutorImpl.java:67)
	at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:422)
	at org.sonar.server.platform.db.migration.step.MigrationStepsExecutorImpl.execute(MigrationStepsExecutorImpl.java:52)
	at org.sonar.server.platform.db.migration.engine.MigrationEngineImpl.execute(MigrationEngineImpl.java:54)
	at org.sonar.server.platform.db.migration.DatabaseMigrationImpl.doUpgradeDb(DatabaseMigrationImpl.java:105)
	at org.sonar.server.platform.db.migration.DatabaseMigrationImpl.doDatabaseMigration(DatabaseMigrationImpl.java:80)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.IllegalStateException: Fail to execute ALTER TABLE components DROP COLUMN module_uuid
	at org.sonar.server.platform.db.migration.step.DdlChange$ContextImpl.execute(DdlChange.java:106)
	at org.sonar.server.platform.db.migration.step.DdlChange$ContextImpl.execute(DdlChange.java:86)
	at org.sonar.server.platform.db.migration.step.DdlChange$ContextImpl.execute(DdlChange.java:128)
	at org.sonar.server.platform.db.migration.step.DropColumnChange.execute(DropColumnChange.java:49)
	at org.sonar.server.platform.db.migration.step.DdlChange.execute(DdlChange.java:45)
	at org.sonar.server.platform.db.migration.step.MigrationStepsExecutorImpl.execute(MigrationStepsExecutorImpl.java:75)
	... 9 common frames omitted
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The index 'IX_components__branch_uuid' is dependent on column 'module_uuid'.
	at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:259)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1695)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:920)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:814)
	at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7675)
	at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:4137)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:272)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:246)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.execute(SQLServerStatement.java:787)
	at com.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:94)
	at com.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java)
	at org.sonar.server.platform.db.migration.step.DdlChange$ContextImpl.execute(DdlChange.java:91)
	... 14 common frames omitted

The last thing that can be seen is this:
grafik

Thanks in advance.
Thomas

Colin · April 11, 2024, 2:35pm

Hey there

Could a rogue DBA have added this index? You might want to manually drop it from the database, and see if the upgrade proceeds.

Normally, no alterations should be made to the database (like adding an index) outside of the upgrade process.

ThE · April 12, 2024, 12:20pm

Hi Collin,

we deleted the index, and the colomn got dropped successfully.
Why did this happen to the database, as far as I know we’ve done no manual modifications.

Yet we try and error the upgrade by removing the “blocking” index iteratively.

org.sonar.server.platform.db.migration.step.MigrationStepExecutionException: Execution of migration step #100002 'Drop column 'module_uuid' in the 'Components' table' failed
...
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The index 'IX_components__branch_uuid' is dependent on column 'module_uuid'.

org.sonar.server.platform.db.migration.step.MigrationStepExecutionException: Execution of migration step #102024 'Drop 'purge_status' column in 'snapshots' table' failed
...
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The index 'IX_snapshots__component_uuid_islast_purge_status' is dependent on column 'purge_status'.

After we deleted those two indices the Database was up-tp-date. But still the SonarQube Server is not starting:

2024.04.12 14:17:26 INFO  web[][o.s.s.p.DetectPluginChange] Detect plugin changes
2024.04.12 14:17:26 INFO  web[][o.s.s.p.DetectPluginChange] No plugin change detected
2024.04.12 14:17:27 INFO  web[][o.h.v.i.util.Version] HV000001: Hibernate Validator null
2024.04.12 14:17:28 WARN  web[][o.s.c.a.AnnotationConfigApplicationContext] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@659e0bfd-org.sonar.server.es.IndexCreator': Initialization of bean failed; nested exception is java.lang.NullPointerException: Cannot invoke "org.elasticsearch.common.document.DocumentField.getValue()" because "field" is null
2024.04.12 14:17:28 ERROR web[][o.s.s.p.Platform] Background initialization failed. Stopping SonarQube
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@659e0bfd-org.sonar.server.es.IndexCreator': Initialization of bean failed; nested exception is java.lang.NullPointerException: Cannot invoke "org.elasticsearch.common.document.DocumentField.getValue()" because "field" is null
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:628)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:920)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:222)
	at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:80)
	at org.sonar.server.platform.platformlevel.PlatformLevel4.start(PlatformLevel4.java:687)
	at org.sonar.server.platform.PlatformImpl.start(PlatformImpl.java:214)
	at org.sonar.server.platform.PlatformImpl.startLevel34Containers(PlatformImpl.java:194)
	at org.sonar.server.platform.PlatformImpl$AutoStarterRunnable.runIfNotAborted(PlatformImpl.java:362)
	at org.sonar.server.platform.PlatformImpl$1.doRun(PlatformImpl.java:113)
	at org.sonar.server.platform.PlatformImpl$AutoStarterRunnable.run(PlatformImpl.java:346)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.NullPointerException: Cannot invoke "org.elasticsearch.common.document.DocumentField.getValue()" because "field" is null
	at org.sonar.server.es.metadata.MetadataIndexImpl.getMetadata(MetadataIndexImpl.java:97)
	at org.sonar.server.es.metadata.MetadataIndexImpl.getDbVendor(MetadataIndexImpl.java:83)
	at org.sonar.server.es.metadata.EsDbCompatibilityImpl.hasSameDbVendor(EsDbCompatibilityImpl.java:37)
	at org.sonar.server.es.IndexCreator.checkDbCompatibility(IndexCreator.java:183)
	at org.sonar.server.es.IndexCreator.start(IndexCreator.java:87)
	at org.sonar.core.platform.StartableBeanPostProcessor.postProcessBeforeInitialization(StartableBeanPostProcessor.java:33)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:440)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
	... 17 common frames omitted
2024.04.12 14:17:29 INFO  web[][o.s.p.ProcessEntryPoint] Hard stopping process
2024.04.12 14:17:29 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown initiated...
2024.04.12 14:17:29 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown completed.

The issue seems to be known Sonarqube10 deployment error but the resolution does not fit to our usecase.

Thanks in advance
Thomas

AndrewHaine · June 13, 2024, 9:01am

Has there been any solution to this? I’ve experienced the same problem trying to upgrade from 10.1.0.

I created post on this some time ago - SonarQube Developer Edition upgrade from 10.1.0 to higher versions fail) - and after checking the logs as advised, I discovered the same error.

ThE · June 17, 2024, 7:10am

Yet there is no solution.

Since the Database migration suceeded after the two columns got dropped I’ll give it another try with the latest available SQ version in the upcoming days.

ThE · August 6, 2024, 12:29pm

Today we upgraded to sonarqube-developer-10.6.0.92116 without any harm.
We checked the database before the migration for the two indices mentioned above. Both were no longer present.

system · August 13, 2024, 12:30pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.