API query != sonar.newCode.referenceBranch settings

markluebbehuesen · January 12, 2024, 12:23pm

Hi,

We currently use Sonarqube 10.1 developer edition with Jenkins.

Thank you for introducing the Sonarscanner property “sonar.newCode.referenceBranch”. I have also requested and voted for this property, as you can see in this topic below.

We are using this sonarscanner property in every project via jenkinsfile to set the new code period for each branch. Works as expected, see the following picture

sonar_new_code

In the GUI, the New Code period is set as follows. Defaults settings, not touched or changed

sonar_new_code_gui

So, my problem is.

Of course I would also like to use the API to get important information, but unfortunately this is not possible. The query (api/new_code_periods/list? project=abc) via API does not match the New CODE PERIOD set by sonarscanner property. The Information of the query is taken from the GUI. See the following screenshot.

If I use the sonarscanner property, of course the API request should also return the same infos. I think this is a bug.

Are there any plans to change this?

thanks,
Mark

ganncamp · January 16, 2024, 5:51pm

Hi Mark,

Can you see if this is still replicable in the current version, 10.3? (Note that 10.4 is imminent.)

Thx,
Ann

anon67236913 · January 17, 2024, 8:12am

Hi @ganncamp @markluebbehuesen,

i guess it’s fixed in 10.3
For a simple test project with this setting

web ui for a branch != master has

2024-01-17_09h08_27

web api /api/new_code_periods/list?project=org.sonarqube:sonarscanner-maven-basic

{
	"newCodePeriods": [
		{
			"projectKey": "org.sonarqube:sonarscanner-maven-basic",
			"branchKey": "foobar",
			"type": "REFERENCE_BRANCH",
			"value": "master",
			"inherited": true,
			"updatedAt": 1700144901872
		},
		{
			"projectKey": "org.sonarqube:sonarscanner-maven-basic",
			"branchKey": "master",
			"type": "PREVIOUS_VERSION",
			"updatedAt": 1700144902592
		}
	]
}

means the same behaviour as in Sonarqube 9.9.x LTS
I have no version 10.1 installed anymore, only 9.9 and 10.3

Gilbert

markluebbehuesen · January 17, 2024, 12:47pm

Hi @anon67236913 @ganncamp

Thank you for your answers.

Hopefully, I can test it in the next few days.

@anon67236913 : Do I understand your answer correctly? You set the new code period by property and then its automatically set in the GUI? A kind of synchronization?

thx,
Mark

anon67236913 · January 17, 2024, 1:49pm

It’s all done via web api calls wrapped in a Jenkins shared library, the Web UI is not involved.
This shared library is part of a generic Maven multibranch pipeline which itself is a shared library
configured via a pipeline.yaml file - this pipeline is used for nearly all Maven projects.

These are the steps for project creation / configuration

Check for project existence /api/projects/search
if no, create it /api/projects/create
Optionally rename the main branch if set in pipeline.yaml
if a branch with that name already exists, it has to be deleted before rename if not set as main
branch already /api/project_branches/delete
rename main branch to the given branch and set as reference branch for all branches
/api/project_branches/rename
Set main branch as new code reference for all branches
api/new_code_periods/set
Set new code period for main branch to previous version
api/new_code_periods/set
Optionally set a special reference branch other than main branch for the $BRANCH_NAME
if set in pipeline.yaml /api/new_code_periods/set

And yes it is in sync / fits afterwards.

anon67236913 · January 23, 2024, 11:59am

After rereading Mark’s post i recognized he’s not using the web api but uses the property
sonar.newCode.referenceBranch instead.

So i did another test with Sonarqube 10.3 and the latest sonar-maven plugin

initial analysis of master branch
mvn clean verify sonar:sonar -Dsonar.host.url=https://xxx -Dsonar.token=squ_xxx

Project is created with the default new code setting = previous version

initial analysis of branch develop and setting the master as reference branch for new code
via sonar.newCode.referenceBranch=master

mvn clean verify sonar:sonar -Dsonar.host.url=https://xxx -Dsonar.token=squ_xxx
-Dsonar.branch.name=develop -Dsonar.newCode.referenceBranch=master

Branch is created but the new code settings stays default to previous version

The response for the web api call /api/new_code_periods/list is

{
	"newCodePeriods": [
		{
			"projectKey": "xxx.xxx.sonar.custom.java:java-custom-rules",
			"branchKey": "develop",
			"type": "PREVIOUS_VERSION",
			"inherited": true
		},
		{
			"projectKey": "xxx.xxx.sonar.custom.java:java-custom-rules",
			"branchKey": "master",
			"type": "PREVIOUS_VERSION",
			"inherited": true
		}
	]
}

The Sonarqube web ui has

but it should have been

This means it works correctly when using the web api api/new_code_periods/set but it doesn’t
work with the property sonar.newCode.referenceBranch
Subsequent scans of the develop Branch doesn’t change anything, it’s as if the property
is ignored.

So Mark @markluebbehuesen is right, it’s a bug - also with the latest Sonarqube 10.3 !!

ganncamp · January 23, 2024, 1:03pm

Hi,

To be clear, this was your first analysis of this particular branch, right?

Thx,
Ann

anon67236913 · January 23, 2024, 1:18pm

Hi,

yes, it means ‘second step = analysis of the develop branch’ , but i’ve tried also with subsequent analysises, didn’t change anything.
It’s as if the property is ignored completely.

Gilbert

ganncamp · January 23, 2024, 1:35pm

Hi,

Thanks for the confirmation. I’ve flagged this for the product team.

Ann

markluebbehuesen · January 23, 2024, 1:59pm

thanks @ganncamp @anon67236913

anon67236913 · January 23, 2024, 2:46pm

Hi Ann,

i’ve never used it so far, but if i remember correctly the property sonar.newCode.referenceBranch was introduced in Sonarqube 9.4

So to be sure, i did another test with our production instance = Sonarqube 9.9.0
and also here it behaves like in Sonarqube 10.3 - the property don’t work.

api/new_code_periods/list

{
{
	"newCodePeriods": [
		{
			"projectKey": "xxx.xxx.sonar.custom.java:java-custom-rules",
			"branchKey": "Sonarqube_8_9_LTS",
			"type": "PREVIOUS_VERSION",
			"inherited": true
		},
		{
			"projectKey": "xxx.xxxstrong text.sonar.custom.java:java-custom-rules",
			"branchKey": "master",
			"type": "PREVIOUS_VERSION",
			"inherited": true
		}
	]
}

=== EDIT ===

Sorry i was wrong, it behaves exactly as with Sonarqube 10.3

Mixed up pom.xml, then a Git event caused the start of the Jenkins pipeline for the
“xxx.xxx.sonar.custom.java:java-custom-rules” project and this Jenkins pipeline
uses the Sonarqube web api for the new code settings as described in a former posting.
Therefore i assumed at first it would work in Sonarqube 9.9.0

Gilbert

anon67236913 · January 24, 2024, 2:38pm

I wonder why the parameter sonar.newCode.referenceBranch is not included in the submit request ?

mvn command

 mvn -X clean verify sonar:sonar -Dsonar.host.url=https://xxx Dsonar.login=xxx
 -Dsonar.branch.name=develop -Dsonar.newCode.referenceBranch=master

submit request, seen in Maven console log and Sonarqube access.log
"POST /api/ce/submit?projectKey=org.sonarqube%3Asonarscanner-maven-basic&projectName=Example%20of%20basic%20Maven%20project&characteristic=branch%3Ddevelop&characteristic=branchType%3DBRANCH HTTP/1.1" 200 44 "-" "ScannerMaven/3.10.0.2594/3.9.6"

It’s the same with Sonarqube 9.9 LTS and latest Sonarqube 10.3

ganncamp · January 24, 2024, 3:53pm

It’s presumably in the analysis report bundle. The request query string is for routing the bundle, I believe.

Ann

anon67236913 · January 25, 2024, 2:45pm

Test in Sonarqube 10.3
Ok, it’s also seen in the scanner context

2024-01-25_15h31_59

and it works correctly as Mark already mentioned.

It’s just that the Project Settting / New Code page shows the wrong setting
and as a consequence api/new_code_periods/list gives also wrong values.

This should be fixed because it’s confusing.

ganncamp · February 2, 2024, 6:34pm

Hi all,

Okay… I have an answer. You’re not going to like it. I don’t like it either:

This is working as designed. Even when used at branch creation, the value of sonar.newCode.referenceBranch is not persisted.

So if you want to use a New Code definition for your branch that’s non-standard for your project, you’ll need to either use sonar.newCode.referenceBranch at Every. Single. Analysis.

Or you’ll need to update the value via the UI.

Ann

anon67236913 · February 2, 2024, 6:40pm

Hi Ann,

Because normally the reference branch changes with every analysis, right ?

Thank you for bringing up the subject!
@markluebbehuesen this means it’s best to use the web api

Gilbert

markluebbehuesen · February 5, 2024, 9:48am

Hi @ganncamp, @anon67236913,

that makes me a little thoughtful from a developer’s point of view.

We use the property for every analysis/branch. Without this parameter we have to adapt always the UI especially for short-lived branches.

I really don’t understand what the property “sonar.newCode.referenceBranches” was introduced for if it doesn’t work in sync with the UI or Web API.

In my opinion, if you introduce such a parameter, it should also be fully integrated into the system (UI, Web API).
Sorry, but now it’s only half a job.

@ganncamp : Can you please invite a developer of this parameter to this topic? thx

thanks,
Mark

ganncamp · February 5, 2024, 12:33pm

Hi Mark,

The Product Managers are already aware.

Ann