OIDC Okta Authentication is not working after upgrade of sonarqube to v9.9.2

Hello All,

I was upgrading the sonarqube from v8.9.6-developer to v9.9.2-developer using terraform and Teamcity,which was deployed using Kubernetes and also changed the plugins used to the compatible versions of v9.9 according to the plugin matrix.

System Information

* Java version -- openjdk 17.0.8.1
* Database: PostgresSQL -- v13.2 
* Operating System -- Linux
* OIDC Plugin used -- v2.1.1

Issue
As a part of the upgrade and plugin version update, modified the sonar-auth-oidc plugin version to
v2.1.1. For the existing OIDC config only, modified the version. But while login to sonarqube with this URL(https://sonarqube.*****/oauth2/callback/saml), it was saying

You’re not authorized to access this page. Please contact the administrator.

While login using this url (https://sonarqube.****/), it was asking for Login and Password, not showing the OKTA button. Will there be any default credentials for this?

Checked the <SONARQUBE_HOME>/web.log below are the some of log snippets

2023.10.16 18:02:16 WARN  web[][o.s.db.dialect.H2] H2 database should be used for evaluation purpose only.
2023.10.16 18:02:22 WARN  web[][o.s.a.s.w.WebService$Action] Description is not set on action api/monitoring/metrics
2023.10.16 18:02:22 WARN  web[][o.s.a.s.w.WebService$Action] Since is not set on action api/monitoring/metrics
2023.10.16 18:02:22 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/monitoring/metrics
2023.10.16 18:02:22 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/system/liveness
2023.10.16 18:02:25 WARN  web[][o.s.s.a.LogOAuthWarning] For security reasons, OAuth authentication should use HTTPS. You should set the property 'Administration > Configuration > Server base URL' to a HTTPS URL.
2023.10.16 18:02:27 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action saml/validation_init
2023.10.16 18:02:27 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/system/liveness
2023.10.16 18:02:27 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/plugins/download
2023.10.16 18:02:27 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/analysis_cache/get
2023.10.16 18:02:27 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/alm_integrations/check_pat
2023.10.16 18:02:27 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/push/sonarlint_events
2023.10.16 18:02:27 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/support/info
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.sonar.process.PluginSecurityManager (file:/opt/sonarqube/lib/sonar-application-9.9.2.77730.jar)
WARNING: Please consider reporting this to the maintainers of org.sonar.process.PluginSecurityManager
WARNING: System::setSecurityManager will be removed in a future release
2023.10.16 18:02:28 INFO  ce[][o.s.p.ProcessEntryPoint] Starting Compute Engine
2023.10.16 18:02:28 INFO  ce[][o.s.ce.app.CeServer] Compute Engine starting up...
2023.10.16 18:02:29 INFO  ce[][o.sonar.db.Database] Create JDBC data source for jdbc:h2:tcp:******:9092/sonar;NON_KEYWORDS=VALUE
2023.10.16 18:02:29 INFO  ce[][c.z.h.HikariDataSource] HikariPool-1 - Starting...
2023.10.16 18:02:29 INFO  ce[][c.z.h.p.HikariPool] HikariPool-1 - Added connection conn0: url=jdbc:h2:tcp://*****/sonar user=
2023.10.16 18:02:29 INFO  ce[][c.z.h.HikariDataSource] HikariPool-1 - Start completed.
2023.10.16 18:02:29 WARN  ce[][o.s.db.dialect.H2] H2 database should be used for evaluation purpose only.
2023.10.16 18:02:31 INFO  ce[][o.s.s.p.ServerFileSystemImpl] SonarQube home: /opt/sonarqube
2023.10.16 18:02:31 INFO  ce[][o.s.c.e.CoreExtensionsLoader] Loaded core extensions: developer-edition, developer-scanner, developer-server, license, monitoring
2023.10.16 18:02:32 INFO  ce[][o.s.c.c.CePluginRepository] Load plugins
2023.10.16 18:02:35 INFO  ce[][o.s.c.c.ComputeEngineContainerImpl] Running Developer edition
2023.10.16 18:02:35 INFO  ce[][o.s.ce.app.CeServer] Compute Engine is started
2023.10.16 18:02:35 INFO  app[][o.s.a.SchedulerImpl] Process[ce] is up
2023.10.16 18:02:35 INFO  app[][o.s.a.SchedulerImpl] SonarQube is operational
java.lang.IllegalArgumentException: Identity provider oidc does not exist or is not enabled
	at org.sonar.server.authentication.IdentityProviderRepository.getEnabledByKey(IdentityProviderRepository.java:48)
	at org.sonar.server.authentication.AuthenticationFilter.resolveProviderOrHandleResponse(AuthenticationFilter.java:54)
	at org.sonar.server.authentication.OAuth2CallbackFilter.doFilter(OAuth2CallbackFilter.java:66)
	at org.sonar.server.platform.web.MasterServletFilter$GodFilterChain.doFilter(MasterServletFilter.java:153)
	at org.sonar.server.authentication.DefaultAdminCredentialsVerifierFilter.doFilter(DefaultAdminCredentialsVerifierFilter.java:89)
	at org.sonar.server.platform.web.MasterServletFilter$GodFilterChain.doFilter(MasterServletFilter.java:153)
	at org.sonar.server.platform.web.MasterServletFilter.doFilter(MasterServletFilter.java:116)
	at jdk.internal.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)

There are no errors in es.log and also verified that sonarqube pod is in running state. The OIDC Auth was working as expected with sonarqube v8.9 and OIDC v1.1.0

And the response for this url https://<your_sonarqube_url>/api/users/identity_providers is empty
{"identityProviders":[]}

Can you please assist me on this issue.

Please let me know for any additional info.

Thanks

Hey there.

Troubleshooting GitHub - vaulttec/sonar-auth-oidc: OpenID Connect (OIDC) Plugin for SonarQube is probably best done by raising an issue with the maintainer on their Github repo.

Thanks Colin for the info,

Observed that there are some more pre things to check as a apart of v9.9.2 update, those may causing this issue.

In the manifest file of the sonarqube, changed the prefix of all env variables from SONARQUBE to SONAR includes JDBC, Web, Search and CE JavaOpts, and OIDC related variables.

Question : As per the release doc, some env variables which is having prefix SONARQUBE got deprecated, so this means do we need to change all the env variables used in different scripts related the upgrade?

After this change, identified error while doing the healthcheck of ES, but found this error in sonar.log

2023.10.18 19:21:29 DEBUG app[][o.e.c.RestClient] updated [[host=http://127.0.0.1:9001]] already in blacklist
2023.10.18 19:21:29 DEBUG app[][o.a.h.i.n.c.MainClientExec] [exchange: 95] start execution
2023.10.18 19:21:29 DEBUG app[][o.a.h.c.p.RequestAddCookies] CookieSpec selected: default
2023.10.18 19:21:29 DEBUG app[][o.a.h.c.p.RequestAuthCache] Re-using cached 'basic' auth scheme for http://127.0.0.1:9001
2023.10.18 19:21:29 DEBUG app[][o.a.h.c.p.RequestAuthCache] No credentials for preemptive authentication
2023.10.18 19:21:29 DEBUG app[][o.a.h.i.n.c.InternalHttpAsyncClient] [exchange: 95] Request connection for {}->http://127.0.0.1:9001
2023.10.18 19:21:29 DEBUG app[][o.a.h.i.n.c.PoolingNHttpClientConnectionManager] Connection request: [route: {}->http://127.0.0.1:9001][total kept alive: 0; route allocated: 0 of 10; total allocated: 0 of 30]
2023.10.18 19:21:29 DEBUG app[][o.a.h.i.n.c.PoolingNHttpClientConnectionManager] Connection request failed
java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.pollConnect(Native Method)
	at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source)
	at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
	at java.base/java.lang.Thread.run(Unknown Source)
2023.10.18 19:21:29 DEBUG app[][o.a.h.i.n.c.InternalHttpAsyncClient] [exchange: 95] connection request failed
2023.10.18 19:21:29 DEBUG app[][o.e.c.RestClient] request [GET http://127.0.0.1:9001/] failed

Since the ES not proper, the SonarQube also failed.

Also migrated the latest snapshot data to the database using.

Can you please help me with this issue

You can find all valid environment variables here.

Assuming the rest of startup goes fine, this is a DEBUG message safe to ignore.

Thanks Colin, Restarted the ES then worked. Now sonar is running and also able to login okta.

Regarding the env variables, followed the doc you shared but observed minor difference in the name of variable(Property) from the document and Sonarqube source code. For example, the javaopts of ES is referred as SONAR_SEARCH_JAVAADDITIONALOPTS wherein from the property class from the source code it is referred as [SEARCH_JAVA_ADDITIONAL_OPTS](https://github.com/SonarSource/sonarqube/blob/master/server/sonar-process/src/main/java/org/sonar/process/ProcessProperties.java#L87)

Just want to check this difference make any impact?

And also one more , Is there any document where I can find the list of deprecated WebApI endpoints in the Sonar v9.9.2?

You should rely on what is documented – otherwise, you risk misinterpreting the source code (the java property to environment variable conversion is being done here)

You can find information about what has been deprecated in the Web API in the upgrade notes.

Thanks for the info Colin.