SonarQube 8.9 LTS: best practise for retrieving an old Quality profile with pylint rules?

SonarQube Server / Community Build
1

Hello,
I want to migrate from SQ 7.9.3 to SQ 8.9.2.
In my project, developers are using pylint, with a specific pylintrc file.

The previous pylint repository does no longer exist in SQ 8.9, so :

  • How to take in account the error/warning messages from pylint ?

    example: the rule Access of nonexistent member (rule “Pylint:E1101” in pylint repo in SQ 7.9) seems no exist in the repositories of SQ 8.9 (SonarQube and Common Python)

  • What the best procedure for “using” the previous Python Quality profile used in SQ 7.9 ?
    (in addition with Importing Third-Party Issues

Thank you for any support.

2

Hey Tom.

Indeed – in SonarQube v8.9 LTS it is no longer possible to manage Pylint rule sets in SonarQube (checkout MMF-1778 for some more background).

Issues will need to be imported by Importing Third-Party Issues as you’ve already noted. In what is maybe good news – everything configured via a pylintrc file will just work when importing pylint issues as third-party issues (what rules are disabled, files are ignored, etc.)

3

Thank you for the answer.
Do I have to change the text output of pylint in to the json expected format ?
Is there somewhere a tool that automate the task ?
Sorry If I did get the context (I’am not an expert on SonarQube)
eg
pylint output:
src/sample/my_method.py:9: [W0612(unused-variable), func_1] Unused variable 'a' →

According Json third-party issue to build ?

{ "issues": [
    {
      "engineId": "unused-variable",
      "ruleId": "unused-variable)",
      "severity":"MAJOR",
      "type":"CODE_SMELL",
      "primaryLocation": {
        "message": "Unused variable 'a'` ",
        "filePath": "src/sample/my_method.py",
        "textRange": {
          "startLine": 9,
          "endLine": 9,
          "startColumn": 0,
          "endColumn": 10
        }
      },
      "effortMinutes": 5
    }
]
4

Hey there.

You’ll want to use the --output-format=parseable Pylint option before passing the report to the sonar.python.pylint.reportPaths analysis parameter.

5

Thank you for the answer.
I understand that the pylint option –output-format=parseable avoid the low-level option –msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}"
…but the output text file still lines as src/sample/my_method.py:9: [W0612(unused-variable), func_1] Unused variable 'a'…
without indication for SonarQube to know what is the severity of the violated rule (here unused-variable).

I tried to modify manually the xml quality profile file (eg MAJOR → MINOR severity) but I dif not see any change…

		<rule>
			<repositoryKey>external_pylint</repositoryKey><type>CODE_SMELL</type>
			<key>W0612</key>
			<priority>MINOR</priority>
			<parameters/>
		</rule>
6

Hey there.

All imported Pylint issues will have a Major severity, and it cannot be overriden.

7

Thank you for the confirmation. It’s true I read this statement but I believed it was possible to set-up a workaround.
Developers in my project won’t be happy the see their Quality Gates failed (threshold on the major issues…).
Have a good day !

8

I just stumbled across this as well. Having all Pylint issues set to Major is a show stopper for us, since we had most of them on Minor. I think I will have to stop linting with Pylint, since not upgrading SonarQube is not really an option. It’s a real shame the default external issue priority is not configurable.

9

I found a workaround: import the Pylint report as generic issues. With a little bit of magic from jq it’s possible to transform Pylints default JSON output to the format SonarQube expects:

$ SONAR_PYLINT_JSON_FORMAT='{
  engineId: "PYLINT",
  ruleId: .["message-id"],
  type: "CODE_SMELL",
  primaryLocation: {
    message: .message,
    filePath: .path,
    textRange: {
      startLine: .line,
      startColumn: .column
    }
  },
  severity: "MINOR",
  effortMinutes: 5
}'
$ pylint --output-format=json <other_args> \
    | jq ".[] | $SONAR_PYLINT_JSON_FORMAT" | jq -s "{issues: .}"
2 Likes
10

Thank you for the information. I have to test and use it !
Now…if you have an answer to this (my) question (sorry…:slight_smile: )

11

Hello,
(sorry for the delay) I 've tried the solution above and it works !
Thank you very much.
Now, about the severities, it’s not obvious to let SonarQube working like the previous configuration (Sonar 7.9). In my understanding it is not possible to link an external issues to a rule (with its severity) with an existing Quality Profile.
So an intermediate step has to be set-up somewhere.

12

I’ve written a PyLint plugin that exports to a JSON format SonarQube can import, and for which you can easily configure the severity and effort per rule:

Configuring the issue severity is then as easy as

$ pylint \
    --load-plugins=pylint_sonarjson \
    --output-format=sonarjson \
    --sonar-rules=C0114:INFO:10,C0328:MINOR:1 \
    my_file.py
2 Likes