Can't get issues for users with more than 1024 groups

SonarQube
1

In our env, we use GitLab as ALM and we got an error “An error has occurred. Please contact your administrator” on “Issues” tab. The problem is similar to the one described here https://community.sonarsource.com/t/too-many-clauses-maxclausecount-is-set-to-1024/ but in our case user could have more then 1024 groups.

  • versions used SonarQube 8.5.1 (build 38104)
    In web server log:
Caused by: org.elasticsearch.common.io.stream.NotSerializableExceptionWrapper: too_many_clauses: maxClauseCount is set to 1024
	at org.apache.lucene.search.BooleanQuery$Builder.add(BooleanQuery.java:114)

Broken request:

Caused by: org.elasticsearch.index.query.QueryShardException: failed to create query: {
  "bool" : {
    "must" : [
      {
        "match_all" : {
          "boost" : 1.0
        }
      }
    ],
    "filter" : [
      {
        "bool" : {
          "must" : [
            {
              "term" : {
                "indexType" : {
                  "value" : "issue",
                  "boost" : 1.0
                }
              }
            },
            {
              "has_parent" : {
                "query" : {
                  "bool" : {
                    "filter" : [
                      {
                        "bool" : {
                          "should" : [
                            {
                              "term" : {
                                "auth_allowAnyone" : {
                                  "value" : true,
                                  "boost" : 1.0
                                }
                              }
                            },
                            {
                              "term" : {
                                "auth_userIds" : {
                                  "value" : "AXRTfcB-lA3CYH4NGKhe",
                                  "boost" : 1.0
                                }
                              }
                            },
...................
                            {
                              "term" : {
                                "auth_groupIds" : {
                                  "value" : "AXYtqg1T_EIXCvHvjCti",
                                  "boost" : 1.0
                                }
                              }
                            }
                          ],
                          "adjust_pure_negative" : true,
                          "boost" : 1.0
                        }
                      }
                    ],
                    "adjust_pure_negative" : true,
                    "boost" : 1.0
                  }
                },
                "parent_type" : "auth",
                "score" : false,
                "ignore_unmapped" : false,
                "boost" : 1.0
              }
            },
            {
              "bool" : {
                "must_not" : [
                  {
                    "exists" : {
                      "field" : "resolution",
                      "boost" : 1.0
                    }
                  }
                ],
                "adjust_pure_negative" : true,
                "boost" : 1.0
              }
            },
            {
              "term" : {
                "isMainBranch" : {
                  "value" : "true",
                  "boost" : 1.0
                }
              }
            },
            {
              "match_all" : {
                "boost" : 1.0
              }
            }
          ],
          "adjust_pure_negative" : true,
          "boost" : 1.0
        }
      }
    ],
    "adjust_pure_negative" : true,
    "boost" : 1.0
  }
}

Please advise.

3

Hello @Sergei_Poliakov,

The above log extract is not to investigate.
Can you send a full zip of your SonarQube logs directory

Olivier

4

sonar_logs.zip (23.8 KB)
there are logs from our test server

6

Hello @Sergei_Poliakov,

At first sight, it looks like a bug. Give me a bit of time and I’ll come back to you with a definitely answer.
I what circumstances do you have users with more than 1024 groups ?

Olivier

7

Hello @OlivierK,
As I mentioned we use Gitlab as a source of users and groups for SonarQube and we have a lot of projects in Gitlab. Due to the fact that admins are the same in both systems, these users could have membership in ~ 1,3k groups in our environment.

8

Hello @OlivierK,

Do you have any updates on this issue?

9

Hello @Sergei_Poliakov,

Best wishes for 2021. :christmas_tree:
No update on this. Fixing that limitation is unfortunately currently not planned.
I understand that you can have a lot of GitLab projects, but I still don’t get how users can be part of more than 1024 groups. If that’s because each project defines its own groups that sounds like an inefficient (and administratively burdensome) group management strategy :grimacing: .
I am not a GitLab expert but it seems to me that you can use the same groups for several different projects, can’t you ?