Issue with Importing GitHub Projects in 10.3

RyanH · December 14, 2023, 7:29pm

We recently upgraded to SonarQube Enterprise 10.3 (build 82913), and use the Helm chart in a k8s cluster. We use Okta/SAML/SCIM to manage users/authentication.

We have identified a regression in 10.3 with importing new projects from GitHub. After the project is created, only the creator has read-only access to the project, however not even Admin users can access or modify the project.

This is an example of an attempt to perform an API request with an Admin user against the projects created by a different project:

curl -u "$TOKEN:" https://REDACTED/api/projects/update_visibility -X POST -d 'project=my-project_AYw-T2dt8eIyTuaDw4FT&visibility=public'
{"errors":[{"msg":"Insufficient privileges"}]}

Attempting to import the project again reveals that the project has already been imported, but is not visible to any user other than the original creator. However, creating a new “local” and then manually configuring the “DevOps” settings for the project does not have this issue; this suggests the problem is not with the permissions templates (and indeed I have confirmed the templates have the correct permissions.)

Here are the step to reproduce the behavior:

Click “Create Project” and from the drop-down select “From GitHub.”
Select your organization.
Select a repo and click “Import.”
Choose “Use the global setting” and click “Create project.”
Have another user attempt to access or even find the project in the available projects. Similarly, you will also find that even an Admin user cannot modify the configuration/settings for newly created project.

RyanH · December 14, 2023, 7:56pm

I just attempted to reproduce this on our test instance, and it didn’t have the issue. I am not sure why one instance we can import projects, and the other we cannot.

RyanH · December 19, 2023, 12:57am

We still have this issue in our production environment. Does anyone have any ideas on how we can debug this issue?

Colin · December 20, 2023, 8:12am

Hey there.

Sorry this got lost!

Do you have Automatic user, group, and permission provisioning enabled, including Provision project visibility?

This can be found in your global Administration > Authentication > GitHub

smanning · January 5, 2024, 8:19pm

We are having a similar issue to what Ryan describes, which started occurring after our upgrade to SonarQube Enterprise 10.3. This is happening on both our test and production SonarQube instances. We do not have automatic user, group, and permission provisioning enabled with GitHub. We use SAML (via Pingfederate) to manage user authentication. We do have a DevOps platform integration configured with our self-hosted GitHub instances. Our Default visibility of new projects is set to Public.

This is the testing I’ve completed on our test instance and the results I’m seeing:

Created a new public GitHub repository named sonarqube-test. A SonarQube scan is run via a GitHub Actions workflow step. For this example, the SonarScanner for Gradle plugin is being used
A SonarQube project for sonarqube-test didn’t exist yet, so a new one was created
- Our Default permissions template for projects is applied, granting the expected access
- Problem: the Project visibility is set to PRIVATE, not the default PUBLIC

I enabled TRACE logging for this testing to get as much info as possible and have included what appear to be the relevant logs for this project creation. I redacted some info, replacing it with: {redacted description}.

Web.log:

2024.01.05 15:46:00 INFO web[{redacted id}][o.s.s.a.w.GithubProjectCreatorFactory] DevOps configuration SonarQube auto-detected for project {redacted owner}/sonarqube-test

2024.01.05 15:46:00 TRACE web[{redacted id}][sql] time=10ms | sql=INSERT INTO components ( kee, uuid, uuid_path, branch_uuid, name, long_name, qualifier, scope, language, description, private, path, copy_component_uuid, enabled, created_at, b_changed, deprecated_kee, b_copy_component_uuid, b_description, b_enabled, b_language, b_long_name, b_name, b_path, b_qualifier ) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0, null, null, null, 0, null, null, null, null, null ) | params=com.{redacted namespace}:sonarqube-test, {uuid}, {uuid_path}, {branch_uuid}, sonarqube-test, sonarqube-test, TRK, PRJ, 12, 12, true, 12, 12, true, 2024-01-05 15:46:00.356

2024.01.05 15:46:00 TRACE web[{redacted id}][sql] time=5ms | sql=INSERT INTO projects ( kee, qualifier, uuid, name, description, private, tags, creation_method, created_at, updated_at ) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ? ) | params=com.{redacted namespace}:sonarqube-test, TRK, {uuid}, sonarqube-test, 12, true, 12, SCANNER_API_DEVOPS_AUTO_CONFIG, 1704469560356, 1704469560356

Access.log:

{redacted IP} - - [05/Jan/2024:15:46:01 +0000] “POST /api/ce/submit?projectKey=com.{redacted namespace}%3Asonarqube-test&projectName=sonarqube-test&characteristic=devOpsPlatformUrl%3Dhttps%3A%2F%2F{redacted github url}%2Fapi%2Fv3&characteristic=devOpsPlatformProjectIdentifier%3D{redacted owner}%2Fsonarqube-test&characteristic=branch%3Dmaster&characteristic=branchType%3DBRANCH HTTP/1.1” 200 44 “-” “ScannerGradle/4.0.0.2929/Gradle 8.0.2” “{redacted id}”

One thing I noticed is that this log message is new: DevOps configuration SonarQube auto-detected (our GitHub app is named “SonarQube”). I don’t see it present for project creation logs pre-10.3.

I’m assuming that this new log message is related to this change included in the full release notes for 10.3. However I haven’t been able to figure out why/how this is causing SonarQube to create new projects as PRIVATE instead of PUBLIC. Any guidance you can provide on this is appreciated, and please let me know if you need additional information from me. This issue is causing permissions problems with our applications that interact with SonarQube because their access is based on the projects being public.

Colin · January 18, 2024, 9:22am

Hey all.

I’ve referred this topic to some experts. Thanks for the detailed post @smanning!

RyanH · January 18, 2024, 6:55pm

We do have this enabled, but since we are using SAML we don’t appear to have the “Provision project visibility” option.

RyanH · January 18, 2024, 8:40pm

One other thing which might be worth noting is that while I don’t have the issue on our testing environment, I had setup GitHub authentication on my test instance of SonarQube prior to upgrading to try it out and see if we could pull in GitHub Teams for populating groups. I can’t help but wonder if having both auth schemes somehow allowed things to function properly on that instance.

RyanH · January 22, 2024, 11:16pm

@smanning Just checking in and see if you folks have any updates or need any additional information to reproduce the issue.

Thanks!

wojtek.wajerowicz · January 23, 2024, 12:46pm

Hi @RyanH and @smanning,

Thank you for providing the details.

I’ve looked into the issue and it is indeed a bug in SonarQube 10.3.
It is indeed related to auto-binding projects to GitHub DevOps Platform configuration.
If the project is created by running scanner in GitHub Actions and automatically bound to DevOps Platform config project permissions and visibility are not correctly set:

if the instance is managed by SCIM, the visibility is always set to false and the permission templates are not applied( @RyanH’s case)
if the instance is not managed(neither by SCIM nor by syncing users from GitHub), the visibility is set to false and the permission templates are correctly applied(@smanning’s case).

We will fix the issue for the upcoming 10.4 release.

RyanH · January 23, 2024, 5:27pm

@wojtek.wajerowicz

Awesome, thank you!

Will the 10.4 upgrade fix the projects which are in bad states as part of the DB schema upgrade, or will we need run a SQL script to fix these?

wojtek.wajerowicz · January 25, 2024, 7:51am

@RyanH
I don’t think we have enough information in the database about which projects had incorrect permissions/visibility applied and which ones were manually changed.
Fixing this in DB schema migration is not really feasible.

RyanH · January 29, 2024, 10:37pm

That’s understandable.

I suppose what I am driving at is, how do we fix the projects which are currently in this broken state? Will the release notes include documentation on how to do this, or is this something you can help us with or should do before the 10.4 release?

Thanks!

wojtek.wajerowicz · January 30, 2024, 9:01am

Hi @RyanH,

An admin can go to Administration → Projects → Management, select projects that are in an incorrect state, and Bulk Apply Permission Template to update permissions on the selected projects.
After that it should be possible to update the visibility for each project in Project Settings → Permissions.

RyanH · January 31, 2024, 6:39pm

@wojtek.wajerowicz

Ah, thanks! That fixed the broken projects.

system · February 7, 2024, 6:40pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.