SonarCloud with AWS CodeBuild

Viktorf · August 10, 2021, 10:54am

Hello, the command that supposed to test the code with sonar-scanner doesn’t work anymore, due to the lack of a package, I haven’t received this kind of error before, and I don’t know where to look in order to fix it.
The workflow looks like this:
AWS CodePipeline is used to fetch the source code from GitHub, after, source code is passed to CodeBuild project that runs mvn sonar:sonar in order to test the code.

ALM used: GitHub
CI system used AWS CodeBuild
Scanner command used when applicable (private details masked)
mvn sonar:sonar -Dsonar.login=$LOGIN -Dsonar.host.url="https://sonarcloud.io" -Dsonar.projectKey="app-1" -Dsonar.organization="org-name"
Languages of the repository - Java
Steps to reproduce

Create a CodeBuild project, add necessary commands to buildspec.yml (download maven, sonar-scanner, etc.)
Trigger the created CodeBuild project

Error log:

Claire_Villard · August 10, 2021, 2:19pm

Hi @Viktorf , welcome to the community forum.

I’m surprised to see the operation prefix in the error message, while the command uses the sonar prefix.
Is there any plugin on your POM file that uses this prefix?
Here is a Maven documentation about how to define a plugin prefix mapping.

To investigate further, could you please share the full logs and your pom.xml file?

Viktorf · August 10, 2021, 4:07pm

Thanks for your answer.
Interestingly enough it worked just fine a couple of days ago.

Contents of `pom.xml` file:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance"
  xmlns="https://maven.apache.org/POM/4.0.0"
  xsi:schemaLocation="https://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.springframework.samples</groupId>
  <artifactId>spring-petclinic</artifactId>
  <version>2.3.0.BUILD-SNAPSHOT</version>

  <parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.3.0.RC1</version>
  </parent>
  <name>petclinic</name>

  <properties>

    <!-- Generic properties -->
    <java.version>1.8</java.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>

    <!-- Web dependencies -->
    <webjars-bootstrap.version>3.3.6</webjars-bootstrap.version>
    <webjars-jquery-ui.version>1.11.4</webjars-jquery-ui.version>
    <webjars-jquery.version>2.2.4</webjars-jquery.version>
    <wro4j.version>1.8.0</wro4j.version>

    <jacoco.version>0.8.5</jacoco.version>
    <nohttp-checkstyle.version>0.0.4.RELEASE</nohttp-checkstyle.version>
    <spring-format.version>0.0.21</spring-format.version>
  </properties>

  <dependencies>
    <!-- Spring and Spring Boot dependencies -->
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-cache</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-data-jpa</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-validation</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-thymeleaf</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-test</artifactId>
      <scope>test</scope>
        <exclusions>
            <exclusion>
                <groupId>org.junit.vintage</groupId>
                <artifactId>junit-vintage-engine</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <!-- Databases - Uses H2 by default -->
    <dependency>
      <groupId>com.h2database</groupId>
      <artifactId>h2</artifactId>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>mysql</groupId>
      <artifactId>mysql-connector-java</artifactId>
      <scope>runtime</scope>
    </dependency>

    <!-- caching -->
    <dependency>
      <groupId>javax.cache</groupId>
      <artifactId>cache-api</artifactId>
    </dependency>
    <dependency>
      <groupId>org.ehcache</groupId>
      <artifactId>ehcache</artifactId>
    </dependency>

    <!-- webjars -->
    <dependency>
      <groupId>org.webjars</groupId>
      <artifactId>webjars-locator-core</artifactId>
    </dependency>
    <dependency>
      <groupId>org.webjars</groupId>
      <artifactId>jquery</artifactId>
      <version>${webjars-jquery.version}</version>
    </dependency>
    <dependency>
      <groupId>org.webjars</groupId>
      <artifactId>jquery-ui</artifactId>
      <version>${webjars-jquery-ui.version}</version>
    </dependency>
    <dependency>
      <groupId>org.webjars</groupId>
      <artifactId>bootstrap</artifactId>
      <version>${webjars-bootstrap.version}</version>
    </dependency>
    <!-- end of webjars -->

    <!-- Testing -->
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.mockito</groupId>
      <artifactId>mockito-junit-jupiter</artifactId>
      <scope>test</scope>
    </dependency>

    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-devtools</artifactId>
      <optional>true</optional>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>io.spring.javaformat</groupId>
        <artifactId>spring-javaformat-maven-plugin</artifactId>
        <version>${spring-format.version}</version>
        <!-- run ./mvnw spring-javaformat:apply to apply -->
        <executions>
          <execution>
            <phase>validate</phase>
            <goals>
              <goal>validate</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-checkstyle-plugin</artifactId>
        <version>3.1.1</version>
        <dependencies>
          <dependency>
            <groupId>com.puppycrawl.tools</groupId>
            <artifactId>checkstyle</artifactId>
            <version>8.29</version>
          </dependency>
          <dependency>
            <groupId>io.spring.nohttp</groupId>
            <artifactId>nohttp-checkstyle</artifactId>
            <version>${nohttp-checkstyle.version}</version>
          </dependency>
        </dependencies>
        <executions>
          <execution>
            <id>nohttp-checkstyle-validation</id>
            <phase>validate</phase>
            <configuration>
              <configLocation>src/checkstyle/nohttp-checkstyle.xml</configLocation>
              <suppressionsLocation>src/checkstyle/nohttp-checkstyle-suppressions.xml</suppressionsLocation>
              <encoding>UTF-8</encoding>
              <sourceDirectories>${basedir}</sourceDirectories>
              <includes>*</includes>
              <excludes>**/.git/**/*,**/.idea/**/*,**/target/**/,**/.flattened-pom.xml,**/*.class</excludes>
            </configuration>
            <goals>
              <goal>check</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <executions>
          <execution>
            <!-- Spring Boot Actuator displays build-related information
              if a META-INF/build-info.properties file is present -->
            <goals>
              <goal>build-info</goal>
            </goals>
            <configuration>
              <additionalProperties>
                <encoding.source>${project.build.sourceEncoding}</encoding.source>
                <encoding.reporting>${project.reporting.outputEncoding}</encoding.reporting>
                <java.source>${maven.compiler.source}</java.source>
                <java.target>${maven.compiler.target}</java.target>
              </additionalProperties>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.jacoco</groupId>
        <artifactId>jacoco-maven-plugin</artifactId>
        <version>${jacoco.version}</version>
        <executions>
          <execution>
            <goals>
              <goal>prepare-agent</goal>
            </goals>
          </execution>
          <execution>
            <id>report</id>
            <phase>prepare-package</phase>
            <goals>
              <goal>report</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

      <!-- Spring Boot Actuator displays build-related information if a git.properties
        file is present at the classpath -->
      <plugin>
        <groupId>pl.project13.maven</groupId>
        <artifactId>git-commit-id-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>revision</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <verbose>true</verbose>
          <dateFormat>yyyy-MM-dd'T'HH:mm:ssZ</dateFormat>
          <generateGitPropertiesFile>true</generateGitPropertiesFile>
          <generateGitPropertiesFilename>${project.build.outputDirectory}/git.properties
          </generateGitPropertiesFilename>
          <failOnNoGitDirectory>false</failOnNoGitDirectory>
        </configuration>
      </plugin>
      <plugin>
          <groupId>org.sonarsource.scanner.maven</groupId>
          <artifactId>sonar-maven-plugin</artifactId>
          <version>3.7.0.1746</version>
      </plugin>

      <plugin>
        <groupId>ro.isdc.wro4j</groupId>
        <artifactId>wro4j-maven-plugin</artifactId>
        <version>${wro4j.version}</version>
        <executions>
          <execution>
            <phase>generate-resources</phase>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <wroManagerFactory>ro.isdc.wro.maven.plugin.manager.factory.ConfigurableWroManagerFactory</wroManagerFactory>
          <cssDestinationFolder>${project.build.directory}/classes/static/resources/css</cssDestinationFolder>
          <wroFile>${basedir}/src/main/wro/wro.xml</wroFile>
          <extraConfigFile>${basedir}/src/main/wro/wro.properties</extraConfigFile>
          <contextFolder>${basedir}/src/main/less</contextFolder>
        </configuration>
        <dependencies>
          <dependency>
            <groupId>org.webjars</groupId>
            <artifactId>bootstrap</artifactId>
            <version>${webjars-bootstrap.version}</version>
          </dependency>
          <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-core</artifactId>
            <version>${mockito.version}</version>
          </dependency>
        </dependencies>
      </plugin>
    </plugins>
  </build>

  <!-- Apache 2 license -->
  <licenses>
    <license>
      <name>Apache License, Version 2.0</name>
      <url>https://www.apache.org/licenses/LICENSE-2.0</url>
    </license>
  </licenses>

  <repositories>
    <repository>
      <id>spring-snapshots</id>
      <name>Spring Snapshots</name>
      <url>https://repo.spring.io/snapshot</url>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
    </repository>
    <repository>
      <id>spring-milestones</id>
      <name>Spring Milestones</name>
      <url>https://repo.spring.io/milestone</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
    </repository>
  </repositories>

  <pluginRepositories>
    <pluginRepository>
      <id>spring-snapshots</id>
      <name>Spring Snapshots</name>
      <url>https://repo.spring.io/snapshot</url>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
    </pluginRepository>
    <pluginRepository>
      <id>spring-milestones</id>
      <name>Spring Milestones</name>
      <url>https://repo.spring.io/milestone</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
    </pluginRepository>
  </pluginRepositories>

  <profiles>
    <profile>
      <id>m2e</id>
      <activation>
        <property>
          <name>m2e.version</name>
        </property>
      </activation>
      <build>
        <pluginManagement>
          <plugins>
            <!-- This plugin's configuration is used to store Eclipse m2e settings
   only. It has no influence on the Maven build itself. -->
            <plugin>
              <groupId>org.eclipse.m2e</groupId>
              <artifactId>lifecycle-mapping</artifactId>
              <version>1.0.0</version>
              <configuration>
                <lifecycleMappingMetadata>
                  <pluginExecutions>
                    <pluginExecution>
                      <pluginExecutionFilter>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-checkstyle-plugin</artifactId>
                        <versionRange>[1,)</versionRange>
                        <goals>
                          <goal>check</goal>
                        </goals>
                      </pluginExecutionFilter>
                      <action>
                        <ignore/>
                      </action>
                    </pluginExecution>
                    <pluginExecution>
                      <pluginExecutionFilter>
                        <groupId>org.springframework.boot</groupId>
                        <artifactId>spring-boot-maven-plugin</artifactId>
                        <versionRange>[1,)</versionRange>
                        <goals>
                          <goal>build-info</goal>
                        </goals>
                      </pluginExecutionFilter>
                      <action>
                        <ignore/>
                      </action>
                    </pluginExecution>
                  </pluginExecutions>
                </lifecycleMappingMetadata>
              </configuration>
            </plugin>
          </plugins>
        </pluginManagement>
      </build>
    </profile>
  </profiles>
</project>

I don’t know how its best to paste full log file, it contains several thousand lines?

Claire_Villard · August 10, 2021, 4:42pm

Thanks for you POM file. I wasn’t able to reproduce the issue on a local build, an analysis using the same POM and some dummy code ran just fine.
Do you have the issue if you run the exact same command yourself, outside of CodeBuild?

Could you build a minimal reproducer, with the smallest POM and code, that reproduces the issue, and share it here?

Thanks
Claire

Viktorf · August 11, 2021, 9:44am

Thanks for your replies.
Actually, when I ran this code locally it worked.
And when I also created a new CodeBuild project in different AWS account it also worked, now I don’t know what’s the problem.

Claire_Villard · August 11, 2021, 9:58am

On the impacted project, have you tried removing pieces from the POM one by one, to try to identify what piece generate the error?
If you run some other Maven goal instead of sonar:sonar, does it work?

Viktorf · August 11, 2021, 10:01am

I haven’t removed pieces from the POM, I have only added the SonarCloud plugin there.
I have a maven package there, and it works fine.

Claire_Villard · August 11, 2021, 10:07am

Could you please share the Maven logs in debug, and the AWS CodeBuild file without any private detail, using any public file sharing tool you like?

Viktorf · August 11, 2021, 10:13am

buildspec.yml for CodeBuild:

github.com

arthurflame/spring-petclinic/blob/main/buildspec.yml

version: 0.2
phases:
  install:
    runtime-versions:
      java: corretto11
  pre_build:
    commands:
      - apt-get update
      - apt-get install -y jq
      - wget https://www-eu.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz
      - tar xzf apache-maven-3.5.4-bin.tar.gz
      - ln -s apache-maven-3.5.4 maven
      - wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-3.3.0.1492-linux.zip
      - unzip ./sonar-scanner-cli-3.3.0.1492-linux.zip
      - export PATH=$PATH:/sonar-scanner-3.3.0.1492-linux/bin/
  build:
    commands:
#       - mvn test    
      - mvn package
      - mvn sonar:sonar -Dsonar.login=$LOGIN -Dsonar.host.url="https://sonarcloud.io" -Dsonar.projectKey="app-1" -Dsonar.organization="arthurflame"

This file has been truncated. show original

maven.log with debug on:

github.com

arthurflame/spring-petclinic/blob/main/maven.log

[DEBUG]   Imported: org.eclipse.aether.spi < plexus.core



[DEBUG]   Imported: org.eclipse.aether.transfer < plexus.core



[DEBUG]   Imported: org.eclipse.aether.version < plexus.core



[DEBUG]   Imported: org.fusesource.jansi.* < plexus.core



[DEBUG]   Imported: org.slf4j.* < plexus.core

This file has been truncated. show original

Sorry for the formatting.

Thank you

Claire_Villard · August 11, 2021, 12:19pm

I couldn’t reproduce the issue, even when running on CodeBuild with the build spec provided in the sample repo (thanks for that). The build succeeded and the analysis got submitted to SonarCloud.
I used a standard AWS image aws/codebuild/standard:4.0, and a Linux environment.

I notice some download errors in the logs. Since Maven caches the repository failures and do not retries them for some time, is it possible some downloads failed previously due to some transient network errors for example, making the next attempts fail too? Is the error still reproducible with the same CodeBuild project, but a clean Maven repository?

If it is possible, I would suggest deleting the CodeBuild project and re-creating it, to eliminate any weird state.

Viktorf · August 11, 2021, 12:28pm

Thank you, stumbled upon another error:

Will try recreating the CodeBuild project, thank you very much!

Claire_Villard · August 11, 2021, 12:30pm

I had this error once while doing my tests, the cause was a bad SonarCloud token because I run your command blindly without any configuration. With the appropriate token, it succeeded.

Viktorf · August 11, 2021, 1:17pm

Recreated the CodeBuild project with aws/codebuild/standard:4.0 AWS image, running in the same error, token value is correct

Also noticed this warning:

Claire_Villard · August 11, 2021, 2:04pm

Is the token generated from an account that has the Execute Analysis permission on the project?
Is it correctly filled into the sonar.login property?

Viktorf · August 11, 2021, 2:07pm

Yeah, everything is set up properly, although I am storing the token in AWS Parameter Store and passing it to the CodeBuild project, trying to hardcode the token now to see if it works.

Viktorf · August 12, 2021, 8:10am

I have found out, that mvn sonar:sonar is trying to run tests using SonarQube instead of using SonarCloud:

Is it possible to specify SonarCloud for usage instead of SonarQube?

Viktorf · August 12, 2021, 9:38am

Also, when trying to hardcode the Sonar Token in buildspec.yml file, it doesn’t resolve in the CodeBuild:

Claire_Villard · August 12, 2021, 10:15am

Thanks for the investigation.

I have found out, that mvn sonar:sonar is trying to run tests using SonarQube instead of using SonarCloud

This is just a (misleading) log issue, if the URL is set to https://sonarcloud.io/, then it’s fine.

About the authorization issue, can you analyze the project outside of CodeBuild, with the same token, and confirm it works?
Would you also check the organization and project key are correct?
I tried my own CodeBuild project using default parameters and the latest commit on the public repository, it correctly submitted the analysis to SonarCloud without any error
https://sonarcloud.io/summary/overall?id=app-1

Viktorf · August 12, 2021, 10:30am

Organization id is correct, project key is just the name of the project that will be created (I assume, any value passed to project key would be fine).
I have tried running it locally with the token, and it worked just fine (as well it worked just fine, when I created another CodeBuild project and passed in the SonarCloud token directly to buildspec.yml).
But once, I’ve put the token in AWS Parameter Store, it is unable to resolve(?) it, even though the parameter exists and the token value is correct, and since the token doesn’t resolve it assumes that I am using SonarQube(?).
Right now, trying to remove the environment value of parameter-store from buildspec.yml file, and see if it works without it.

Viktorf · August 12, 2021, 10:35am

So I commented out the parameter-store block from buildspec.yml and passed the token directly:

Still having this issue:

Assuming it tries to run it with SonarQube(?)

Topic		Replies	Views
Sonar Scanner Execution Failed in AWS Codebuild SonarQube Server / Community Build	3	3145	May 26, 2020
Error in Travis Log: "You're not authorized to run analysis. Please contact the project administrator" and Travis CI org not sending Jacoco reports to SonarCloud SonarQube Cloud iam , sonarqube-cloud	46	5438	July 31, 2020
Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.7.0.1746:sonar (default-cli) on project : Not authorized. Please check the properties sonar.login and sonar.password. -> [Help 1] SonarQube Server / Community Build sonarqube , maven , scanner , jenkins	3	10748	April 17, 2020
Failed to execute goal. Should I modify the pom? SonarQube Cloud	7	1151	September 22, 2021
Sonarqube java code analysis in AWS Codebuild SonarQube Server / Community Build sonarqube , scanner	1	2299	December 6, 2019

SonarCloud with AWS CodeBuild

Contents of pom.xml file:

I don’t know how its best to paste full log file, it contains several thousand lines?

Sorry for the formatting.

Related topics

Contents of `pom.xml` file: