Orchestrator: Adding support for downloading artifacts without Jfrog

alain.kermis · February 8, 2024, 3:18pm

Good day, SonarSource Community!

As many of you are aware, since 24 January 24, community plugins can no longer make requests to JFrog via the Orchestrator library. This is due to anonymous access being disabled on JFrog: see SONAR-21476.

As of 2024-02-07T23:00:00Z, a fix has been introduced whereby users can now bypass JFrog entirely by doing the following:

Upgrade Orchestrator version to 4.7.1.1872
Utilize OrchestratorBuilder#setOrchestratorProperty(...) to override the default orchestrator.artifactory.url to central maven (or any other maven repository)
All should work after this

Please note: due to these changes, only the following aliases will work when attempting to download artifacts through Orchestrator:

LATEST_RELEASE for the latest release (in terms of version number, not date)
LATEST_RELEASE[x.y] for the latest release of a series, for example LATEST_RELEASE[5.2]
x.y.z for an exact release including build number

Please refer to the README.md for any other information.

Regards
Alain

felipebz · February 8, 2024, 5:13pm

Hi,

Thanks for the update.

I’m not sure if I should open a new topic or not, but… Unfortunately, this isn’t working anymore because the version numbers are ordered using Comparator.naturalOrder(), which is not the correct way to order them:

github.com/SonarSource/orchestrator

sonar-orchestrator/src/main/java/com/sonar/orchestrator/util/MavenVersionResolver.java

cb5c55dbc


      
          MavenRepositoryVersion mavenMetadata = downloadVersions();
          this.versions = mavenMetadata.getVersioning().getVersions();
          this.versions.sort(Comparator.naturalOrder());

As shown in the image below, this is the result after versions.sort:

As result, the getLatestVersion function will consistently return the latest 9.9.x version (currently 9.9.4.87374) instead of the latest 10.x release.

alain.kermis · February 9, 2024, 8:34am

You are indeed correct. This is a bug and a ticket has been created to deal with it. In the meantime, I suggest using LATEST_RELEASE[10] as this will work.

Many thanks for the find!

Alain

alain.kermis · February 9, 2024, 2:10pm

Hi there, @felipebz

The fix has been merged and released. Please use 4.7.1.1872 instead.

Regards

xdwangwei · December 7, 2024, 1:04pm

401 Error While Building SonarQube 7.8 Source Code on macOS 13.4

Hello everyone, I am trying to build the SonarQube 7.8 source code on macOS 13.4. I am using the command ./gradlew build, but I am encountering a 401 error. The specific error message is as follows:

* What went wrong:
A problem occurred configuring root project 'sonarqube'.
> Could not resolve all artifacts for configuration ':classpath'.
   > Could not resolve com.github.ben-manes:gradle-versions-plugin:0.21.0.
     Required by:
         project :
      > Could not resolve com.github.ben-manes:gradle-versions-plugin:0.21.0.
         > Could not get resource 'https://repox.jfrog.io/repox/plugins.gradle.org/com/github/ben-manes/gradle-versions-plugin/0.21.0/gradle-versions-plugin-0.21.0.pom'.
            > Could not HEAD 'https://repox.jfrog.io/repox/plugins.gradle.org/com/github/ben-manes/gradle-versions-plugin/0.21.0/gradle-versions-plugin-0.21.0.pom'. Received status code 401 from server:

I saw this post in the community: Orchestrator: Adding support for downloading artifacts without Jfrog, but I am not sure how to proceed with the steps mentioned. I am not very familiar with SonarQube, so could someone provide more detailed instructions?

My goal is to modify some code in sonar-scanner-engine and re-generate the sonar-scanner-engine-shaded JAR file. Any help would be greatly appreciated!

Thank you all!

robert.zoy · March 15, 2025, 2:51am

hi Vivi,
did you solved your issuse?

robert.zoy · March 15, 2025, 10:06am

hi Alain,
I follow your steps，but it doesn’t work. can you make it sence?
for example,you said

Utilize OrchestratorBuilder#setOrchestratorProperty(...) to override the default orchestrator.artifactory.url to central maven (or any other maven repository)

I set setOrchestratorProperty in ApiDefinitionDownloader.java file. is it correct?

public class ApiDefinitionDownloader {

  public static void main(String[] args) {
    System.out.println(downloadApiDefinition());
  }

  public static String downloadApiDefinition() {
    OrchestratorBuilder builder = Orchestrator.builderEnv()
      .defaultForceAuthentication();
    builder.setEdition(COMMUNITY);
    builder.setZipFile(FileLocation.byWildcardMavenFilename(new File("../sonar-application/build/distributions"), "sonar-application-*.zip").getFile())
      .setOrchestratorProperty("orchestrator.workspaceDir", "build")
      .setOrchestratorProperty("orchestrator.artifactory.url ", "https://repo1.maven.org/maven2/");
    Orchestrator orchestrator = builder.setServerProperty("sonar.forceAuthentication", "false")
      .build();

    orchestrator.start();
    try {
      HttpCall httpCall = orchestrator.getServer().newHttpCall("api/webservices/list").setParam("include_internals", "true");
      HttpResponse response = httpCall.execute();
      return response.getBodyAsString();
    } finally {
      orchestrator.stop();
    }
  }
}

Thanks

alain.kermis · March 17, 2025, 1:44pm

Hi there @robert.zoy

Welcome to the Community!

May I ask for:

The version of orchestrator that you are using?
The error you are getting

Regards

alain.kermis · March 17, 2025, 1:48pm

@xdwangwei ,

Hi there! Did you try with a new version of SQ? 7.8 is quite old. Please refer to the official website for the latest versions.

Kind regards

robert.zoy · March 18, 2025, 8:05am

Hi Alian,

1.The version of orchestrator is 4.7.1.1872,the config in build.gradle is

      dependency 'org.sonarsource.orchestrator:sonar-orchestrator:4.7.1.1872'

2.The specific error I encountered is as follows. After executing ./gradlew build,

AILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':sonar-plugin-api:shadowJar'.
> Could not resolve all dependencies for configuration ':sonar-plugin-api:detachedConfiguration1'.
   > Could not resolve com.fasterxml.jackson:jackson-bom:2.13.2.20220328.
     Required by:
         project :sonar-plugin-api
      > Could not resolve com.fasterxml.jackson:jackson-bom:2.13.2.20220328.
         > Could not get resource 'https://repox.jfrog.io/repox/public/com/fasterxml/jackson/jackson-bom/2.13.2.20220328/jackson-bom-2.13.2.20220328.pom'.
            > Could not GET 'https://repox.jfrog.io/repox/public/com/fasterxml/jackson/jackson-bom/2.13.2.20220328/jackson-bom-2.13.2.20220328.pom'. Received status code 401 from server: 

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

By the way the version of sonarqube source code is 8.9.10.61524

Thanks for your reply

Colin · March 18, 2025, 9:18am

Hey @robert.zoy

The orchestrator has nothing to do with the dependencies being downloaded—you’re facing an error with Gradle simply downloading the dependencies as part of the build (the orchestrator is used to run tests).

I doubt we made any specific effort at the time of SonarQube v8.9’s release (May 2021) to make it easy to build outside of Sonar, and we’re not in a particular rush to make this old release buildable from the source code.

Recent releases should be in a much better state – (./gradlew build works out of the box if you download the source code of 25.3).

If I had to suggest a starting point, it would be to remove any references to artifactory or repox from the settings.gradle, gradle.properties, and build.gradle files!

robert.zoy · March 18, 2025, 12:24pm

Hi Colin,

Thank you sooooo much for your reply. I took your advice and successfully built the project using version 25.3. I will switch my service to the latest version. Thanks again for your help!

Robert

xdwangwei · March 25, 2025, 2:35am

Not yet resolved. Do you have any suggestions or ideas that might help?

xdwangwei · March 25, 2025, 2:38am

Thanks for the suggestion! However, some of our projects are restricted to JDK 8 syntax. Since newer SonarQube versions require at least JDK 17, we have to stick with an older version for compatibility. Appreciate your input though!

Colin · March 25, 2025, 12:28pm

Hey @xdwangwei

This is a misundestanding. The latest version of SonarQube is perfectly happy to anlayze Java 8 code, but the analysis must be spawned by Java 17 or newer. See the docs.

The requirement on the Java runtime environment refers only to the version of Java used by the scanner itself to run. It does not restrict the versions of Java that can be analyzed by the scanner. In addition, the required version changes with successive versions of the scanner.

xdwangwei · April 12, 2025, 12:09pm

Thank you for the clarification! I’ll make sure to try that out.

Regarding upgrading from SonarQube 7.8 to 9.9 with database migration from MySQL to PostgreSQL, could you please advise the recommended migration path? The official documentation mentions PostgreSQL is required since version 8.9, but I’d appreciate guidance on how to transition the existing MySQL data to PostgreSQL during this major version upgrade.

I noticed the upgrade guide recommends sequential version upgrades. Should I first upgrade from 7.8 to intermediate versions (like 8.x) before migrating to PostgreSQL, or is there a direct migration method available?

Colin · April 14, 2025, 8:24am

Hey @xdwangwei

You must go

7.8 → 7.9 → 8.9 → 9.9 → 24.12 → 25.3

The max version of SonarQube that you can use the mysql-migrator with is v7.8, so you should migrate to PostgreSQL with your current version!

Topic		Replies	Views
Community plugins can no longer make requests to jfrog via the Orchestrator library? Plugin Development	10	706	February 23, 2024
Sonar Maven plugin fails at library download SonarQube Server / Community Build maven , jenkins	6	4504	October 22, 2019
Change SonarQube repository address for Maven analysis SonarQube Server / Community Build scanner	0	535	June 12, 2019
Sonar-java-plugin latest version 6.5.1.22586 broken (HTTP 401 Unauthorized response) SonarQube Server / Community Build java , maven , gradle , sonarqube-ide	1	637	July 6, 2020
Sonarqube opensource application list link Suggest new features sonarqube	1	326	November 29, 2019

Orchestrator: Adding support for downloading artifacts without Jfrog

Related topics