# Bug report: authenticated requests get 403 “Insufficient privileges” on every RBAC-checked endpoint, even for the built-in `admin` superuser with correct DB-level permissions
## Environment
- **SonarQube:** Community Build **26.8.0.126808**, self-hosted, Docker image `sonarqube:community`
- **Upgrade path:** 24.12.0.100206 → 25.12.0.117093 → 26.8.0.126808, completed **2026-08-07** (each step followed the documented procedure: clear ES data dir, migrate DB, wait for UP)
- **Database:** PostgreSQL 13 (Docker), schema migrated successfully at each upgrade step, `api/system/status` reports `UP` with no `DB_MIGRATION_NEEDED` pending
- **OS:** Ubuntu 20.04 (host)
- **Reverse proxy:** nginx/Cloudflare in front of `sonar.paymaxi.com`, confirmed *not* interfering (see evidence — 401/403/200 all pass through with correct, well-formed JSON bodies)
- **Relevant setting:** `sonar.forceAuthentication = true` (normal state)
- **Plugins:** all installed plugins are official SonarSource-branded (“X Code Quality and Security”), none flagged incompatible after upgrade
## Summary
Since the 24.12→25.12→26.8 upgrade, **every authenticated request** (HTTP Basic Auth with a user token, or Bearer token — both tried) to an RBAC-protected endpoint returns:
```json
{“errors”:[{“msg”:“Insufficient privileges”}]}
```
with HTTP 403 — **regardless of what permissions are actually granted** in the database, and regardless of whether the token belongs to a `PROJECT_ANALYSIS_TOKEN`, or the built-in `admin` account itself.
Meanwhile, a fully **anonymous** request (no `Authorization` header at all) to the *same* endpoint, for the *same* public project, succeeds normally — but only once `sonar.forceAuthentication` is temporarily set to `false` (with it `true`, anonymous requests correctly get 401, as expected, and authenticated requests still get 403).
In other words: nobody who logs in or presents a token can access anything that requires a permission check. Only fully anonymous access (when allowed) works.
## Impact
Every CI pipeline using a SonarQube token (GitLab CI in our case, `sonar-scanner-cli:11`/CLI 7.3.0) fails at report submission with:
```
ERROR You’re not authorized to analyze this project or the project doesn’t exist on SonarQube and you’re not authorized to create it. Please contact an administrator.
```
This affects **every project on the instance**, confirmed across two unrelated GitLab groups/orgs sharing this SonarQube server.
## Steps to reproduce
1. On a fresh 26.8.0.126808 instance (or one upgraded via 24.12→25.12→26.8), take any project with `private = false` (public) — call its key `PROJECT_KEY`.
2. Confirm in the DB that `group_roles` / `user_roles` grant the expected roles (e.g. `sonar-users` → `user`, `codeviewer` at minimum), and that the built-in `admin` user is a member of `sonar-administrators` (which has global `admin` role in `group_roles` with `entity_uuid IS NULL`).
3. Generate a `PROJECT_ANALYSIS_TOKEN` for `admin` scoped to `PROJECT_KEY` (or use any existing user token).
4. `curl -s -u “:” “https:///api/users/current”` → returns 200 with correct identity, correct `groups` array — but `permissions.global` is an **empty array**, even though `admin` is in a group with global `admin` role.
5. `curl -s -u “:” “https:///api/components/show?component=PROJECT_KEY”` → **403 Insufficient privileges**, even for `admin`, even on `admin`'s own project.
6. `curl -s -u “:” “https:///api/project_analyses/search?project=PROJECT_KEY”` → same 403.
7. `curl -s “https:///api/components/show?component=PROJECT_KEY”` (no auth at all, anonymous, with `sonar.forceAuthentication` temporarily set to `false`) → **200, correct data returned.**
## Expected behavior
A user/token with the DB-recorded permission to browse/analyze a project (via group or direct user role) should be able to do so. At minimum, the `admin` superuser account should never receive “Insufficient privileges” on its own instance.
## Actual behavior
All authenticated (token or login) requests to permission-checked endpoints fail with 403, independent of the actual permission data.
## What we ruled out (with evidence)
We spent significant effort isolating this before concluding it’s an application-level bug, not a config/data/cache issue:
| Hypothesis | Test | Result |
|—|—|—|
| Stale/incorrect DB permissions | Queried `group_roles`, `user_roles`, `groups_users` directly — `sonar-administrators`→`admin` (global), `sonar-users`→`user`/`codeviewer`/`issueadmin`/`securityhotspotadmin` (per-project), no duplicate groups, `admin` correctly linked | **Correct** in DB |
| Missing explicit “Browse” grant on public projects | Backfilled `group_roles` rows for `sonar-users`→`user`,`codeviewer` on 3 affected projects, matching the instance’s own default permission template exactly | **No change** — still 403 |
| In-memory authorization cache stale | Full container restart (`docker compose restart sonarqube`) | **No change** |
| Stale/corrupt ES authorization index | Inspected `components` index `auth_` docs via `docvalue_fields` (`_source` is disabled on this index by mapping) — found `auth_allowAnyone: true` correctly set, `auth_groupIds` empty (expected for a public project) | Data **was already correct** |
| ES index literally not rebuilt | Did a full clear-and-rebuild of the ES data dir (note: the correct dir for this version is `es9`, **not** `es8` — see note below) and confirmed via new index UUIDs (`components` index UUID changed) and `docs.deleted: 0` across all indices that this was a genuine from-scratch reindex from Postgres | **No change** — still 403 with fresh, verified-correct ES data |
| `forceAuthentication` + `allowAnyone` interaction | Temporarily set `sonar.forceAuthentication=false`, restarted. Anonymous request → 200 (correct). **Authenticated** request with the same token → still 403 | Rules out an allowAnyone-specific bug; points to **authenticated-session permission resolution being broken in general** |
| Group-based permission resolution specifically broken (vs. direct user grants) | Inserted a **direct** `user_roles` row (`admin` → `user` role on the project, bypassing groups entirely) | **Still 403** — rules out “just use direct grants instead of groups” as a workaround |
The last two tests are the most conclusive: **anonymous sessions correctly evaluate `allowAnyone`, but any authenticated session — regardless of whether its permission comes from a group or a direct per-user grant — is denied**, even the instance’s own `admin` account. This points to a break in how `UserSession`/authorization resolves permissions specifically for authenticated principals in this build, not a data, cache, or config issue.
## One interesting exception
`GET /api/qualitygates/project_status?projectKey=PROJECT_KEY` (with the same token that gets 403 everywhere else) returns **200** with real, current quality-gate data — suggesting this specific endpoint doesn’t go through the same broken authorization path as `components/show` / `project_analyses/search` / `permissions/users`.
## Side note: `es8` → `es9` directory rename
Our internal runbook (written for the 24.x/25.x line) referenced `/es8/` as the Elasticsearch data directory to clear before a version step. On 26.8.0.126808 the actual directory is `/es9/` — the bundled Elasticsearch major version appears to have moved from 8.x to 9.x in this release (log lines reference `org.elasticsearch.entitlement`, Lucene 99 codec, and other ES9-era internals). Not the root cause of the above (we verified a real es9 rebuild didn’t change the symptom), but worth flagging in the 26.8 upgrade notes/documentation for anyone following an older runbook — `rm -rf es8/*` silently does nothing on 26.8.
## Request
Please advise:
1. Is this a known regression in 26.8.0.126808’s authorization/permission-resolution path?
2. Is there a supported way to fully reset/recompute the authenticated-session permission cache (if one exists beyond what a container restart + ES reindex covers)?
3. Any diagnostic (debug logging, specific class) we can enable to see exactly where the 403 decision is made?
Happy to provide full `sonar.log`/`web.log` excerpts, `group_roles`/`user_roles` dumps, or reproduce anything else needed — everything above was reproduced multiple times, on multiple projects, across two independent GitLab CI consumers of this instance.