Malformed json response reoccurrence

SonarQube Server / Community Build
, ,
1

Hi,

I am facing a bug when initiating a scan on any project on my SonarQube instance

  • Versions used: SQ Version 8.5.1
  • Error observed
##[error]Unhandled Exception:
##[error]Newtonsoft.Json.JsonException: Malformed json response, "actives" field should contain rule 'csharpsquid:S4784'
   at SonarScanner.MSBuild.PreProcessor.SonarWebService.<>c__DisplayClass8_2.<GetActiveRules>b__1(JObject r)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at SonarScanner.MSBuild.PreProcessor.SonarWebService.GetActiveRules(String qprofile)
   at SonarScanner.MSBuild.PreProcessor.TeamBuildPreProcessor.FetchArgumentsAndRulesets(ISonarQubeServer server, ProcessedArgs args, TeamBuildSettings settings, IDictionary`2& serverSettings, List`1& analyzersSettings)
   at SonarScanner.MSBuild.PreProcessor.TeamBuildPreProcessor.DoExecute(ProcessedArgs localSettings)
   at SonarScanner.MSBuild.BootstrapperClass.PreProcess()
   at SonarScanner.MSBuild.BootstrapperClass.Execute()
   at SonarScanner.MSBuild.Program.Execute(String[] args, ILogger logger)
   at SonarScanner.MSBuild.Program.Main(String[] args)
Newtonsoft.Json.JsonException: Malformed json response, "actives" field should contain rule 'csharpsquid:S4784'
   at SonarScanner.MSBuild.PreProcessor.SonarWebService.<>c__DisplayClass8_2.<GetActiveRules>b__1(JObject r)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at SonarScanner.MSBuild.PreProcessor.SonarWebService.GetActiveRules(String qprofile)
   at SonarScanner.MSBuild.PreProcessor.TeamBuildPreProcessor.FetchArgumentsAndRulesets(ISonarQubeServer server, ProcessedArgs args, TeamBuildSettings settings, IDictionary`2& serverSettings, List`1& analyzersSettings)
   at SonarScanner.MSBuild.PreProcessor.TeamBuildPreProcessor.DoExecute(ProcessedArgs localSettings)
   at SonarScanner.MSBuild.BootstrapperClass.PreProcess()
   at SonarScanner.MSBuild.BootstrapperClass.Execute()
   at SonarScanner.MSBuild.Program.Execute(String[] args, ILogger logger)
   at SonarScanner.MSBuild.Program.Main(String[] args)
  • Steps to reproduce N/a
  • Potential workaround N/a

Appears to be a reoccurrence of https://community.sonarsource.com/t/malformed-json-response/5172

Any support on how I can get past this issue would be greatly appreciated.

3

Hello,
Can you make a raw call to the SonarQube API supposed to retrieve the rules active in your C# quality profile (QP for short below). This will 2 requests, one to get the QP key first, and then another one to search all rules actives in that QP. You can use your browser to call the below URLs (and of course a cmd line tool like curl if you prefer). Both Queries are GET

<SONARQUBE_ROOT_URL>/api/qualityprofiles/search?language=cs&qualityProfile=<QP_NAME>

{
    "profiles": [
        {
            "key": "AWIbGczwdy7YnNo8spMl",
            "name": "Sonar way",
            "language": "cs",
            "languageName": "C#",
            "isInherited": false,
            "isDefault": true,
            "activeRuleCount": 257,
            "activeDeprecatedRuleCount": 0,
            "rulesUpdatedAt": "2020-10-05T15:07:49+0000",
            "lastUsed": "2020-05-27T09:55:03+0200",
            "organization": "default-organization",
            "isBuiltIn": true,
            "actions": {
                "edit": false,
                "setAsDefault": false,
                "copy": true,
                "associateProjects": false,
                "delete": false
            }
        }
    ],
    "actions": {
        "create": true
    }
}
# Write down the QP key (AWIbGczwdy7YnNo8spMl above) and reuse in the 2nd query
<SONARQUBE_ROOT_URL>/api/rules/search?activation=true&qprofile=AWIbGczwdy7YnNo8spMl

{
   "total": 399,
   "p": 1,
   "ps": 100,
   "rules": [
      {
         "key": "csharpsquid:S1006",
         "repo": "csharpsquid",
         "name": "Method overrides should not change parameter defaults",
...

Then send me the full result (a JSON result) of the 2nd HTTP(S) request.

Olivier

4

Hi Olivier,

I have carried out the steps as you requested, please see the file attached.

Thanks,
Tom

search.json (468.9 KB)

5

Thanks, and sorry because I forgot 1 key parameter for the 2nd request. I need you to run it again. I also add another to get more data (ps=500).
The correct query is:

<SONARQUBE_ROOT_URL>/api/qualityprofiles/search?language=cs&qualityProfile=<QP_NAME>&f=actives&ps=500

Olivier

6

Hi Olivier,

I have updated the query - please see the updated result below:

search-updated.json (59.2 KB)

7

Hi Tom,

There must be something wrong either in your search index or in your database (why ? I don’t know right now).
Did the problem appear suddenly (without any change on your platform) or after an upgrade ?
If after an upgrade what was the version before upgrade ?

8

Hi Olivier,

I believe the SonarQube instance lost connection to the database, I believe this happened once the connection had been restored.

Is there a way to restore operation or is it a case of creating a clean install or restoring from a backup ?

Thanks,
Tom

9

Hello @tjnashq,

The first thing I would recommend to do is to re-align the DB contents (master data) and the ES (Search) index. To do that:

  • Stop SonarQube
  • Delete the <SONARQUBE_HOME>/data/es6 directory
  • Restart SonarQube. This will trigger a background rebuild of your index from scratch. Projects data will only accessible progressively as they are indexed. The full reindexing can take from a few minutes (if you have a < 5M LoCs) to potentially hours for platforms with several tenths of millions of LoCs

Once SonarQube is restarted (and no need to wait for the projects reindexing) can you retry the very same last query I requested earlier and send me the results again. I should be able to verify if data integrity is OK.

Olivier

10

Hi Olivier,

I have followed the steps to clear that es6 data directory - please see attached the updated file:
search-after-es-reload.json (59.1 KB)

Thank you,
Tom

11

Hello @tjnashq,

The data in the search index seems now correct. The API call should return good data and you should no longer have the original problem.
Let me know.

Olivier

13

Hi Olivier,

I can confirm the error has been resolved - that you for your support with this.

Tom