Sonar analysis takes too much time

SonarCloud
1

Hi,
We have encountered a problem with SonarCloud alaysis. It has became slower and takes more than 20 minutes instead of 2 minutes since Tuesday afternoon. The problem arised 3 days ago and doesn’t want to disappear.

Environment:

  • Azure DevOps git and pipelines.
  • SonarCloud extension for Azure DevOps.
  • Sonar scans .NET 6 project.

I can’t even download the whole log file from Azure DevOps pipeline, I think it is too big. However, in available piece of log I see strange validations, for example:

2023-08-09T21:40:33.8024546Z 21:40:33.404 DEBUG: validate( {"provider":{"metadata":{"name":"WeChat","category":"Social media","message":"Make sure this WeChat key gets revoked, changed, and removed from the code."},"rules":[{"id":"wechat-app-key","rspecKey":"S6652","metadata":{"name":"WeChat app keys should not be disclosed"},"detection":{"matching":{"pattern":"\\bwx[a-f0-9]{16}\\b(?:[^\\r\\n]*?\\r?\\n){0,5}?(?:[^\\r\\n]*?)\\b([a-f0-9]{32})\\b"},"post":{"patternNot":"0{5}|123456","statisticalFilter":{"threshold":3.0}}},"examples":[{"text":"@Schema(description = \"公众号 appId\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"wx8b3a83d0f4efa807\")\n@NotEmpty(message = \"公众号 appId 不能为空\")\nprivate String appId;\n\n@Schema(description = \"公众号密钥\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"40b6b70508b47cbfb4ee39feb617a05a\")\n@NotEmpty(message = \"公众号密钥不能为空\")\n","containsSecret":true,"match":"40b6b70508b47cbfb4ee39feb617a05a"},{"text":"var appId = \"wxdc4e0888857e858d\";\nvar nonceStr = \"2858338391442441211798337075320\";\nvar package = \"prepay_id=wx20171214105006b93f3bb9e90854903185\";\nvar paySign = \"BAF449D858CFD0CE6D488146DEF41A64\";\nvar timeStamp = \"1513219805815\";\n\nwindow.onload = function onBridgeReady(){\n\n    wx.requestPayment({\n        'timeStamp': \"1395712654\",\n        'nonceStr': \"e61463f8efa94090b1f366cccfbbb444\",\n        'package': \"prepay_id=u802345jgfjsdfgsdg888\",\n","containsSecret":false}]}]}}, {"provider":{"metadata":{"name":"WeChat","category":"Social media","message":"Make sure this WeChat key gets revoked, changed, and removed from the code."},"rules":[{"id":"wechat-app-key","rspecKey":"S6652","metadata":{"name":"WeChat app keys should not be disclosed"},"detection":{"matching":{"pattern":"\\bwx[a-f0-9]{16}\\b(?:[^\\r\\n]*?\\r?\\n){0,5}?(?:[^\\r\\n]*?)\\b([a-f0-9]{32})\\b"},"post":{"patternNot":"0{5}|123456","statisticalFilter":{"threshold":3.0}}},"examples":[{"text":"@Schema(description = \"公众号 appId\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"wx8b3a83d0f4efa807\")\n@NotEmpty(message = \"公众号 appId 不能为空\")\nprivate String appId;\n\n@Schema(description = \"公众号密钥\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"40b6b70508b47cbfb4ee39feb617a05a\")\n@NotEmpty(message = \"公众号密钥不能为空\")\n","containsSecret":true,"match":"40b6b70508b47cbfb4ee39feb617a05a"},{"text":"var appId = \"wxdc4e0888857e858d\";\nvar nonceStr = \"2858338391442441211798337075320\";\nvar package = \"prepay_id=wx20171214105006b93f3bb9e90854903185\";\nvar paySign = \"BAF449D858CFD0CE6D488146DEF41A64\";\nvar timeStamp = \"1513219805815\";\n\nwindow.onload = function onBridgeReady(){\n\n    wx.requestPayment({\n        'timeStamp': \"1395712654\",\n        'nonceStr': \"e61463f8efa94090b1f366cccfbbb444\",\n        'package': \"prepay_id=u802345jgfjsdfgsdg888\",\n","containsSecret":false}]}]}}, $)

We don’t have any WeChat integrations in our application.

2

Hi,

Can you provide a bit more of the logs, please?

 
Ann

4

Hi Ann,

Thank you for your response. I have found workaround. I set sonar.verbose=false instead of true. As I understand huge amount of logs flood pipeline. Before all these flooding validation logs I see INFO messages about Sensor. After that I see a lot of DEBUG messages which force pipeline to wait. There were not DEBUG messages before Tuesday afternoon. sonar.verbose=true was set a few monthes ago. More logs:

2023-08-09T21:40:33.8016098Z 21:40:33.119 INFO: Sensor C# Properties [csharp] (done) | time=3ms
2023-08-09T21:40:33.8016430Z 21:40:33.120 INFO: Sensor TextAndSecretsSensor [text]
2023-08-09T21:40:33.8016787Z 21:40:33.120 INFO: Sensor TextAndSecretsSensor is restricted to changed files only
2023-08-09T21:40:33.8024546Z 21:40:33.404 DEBUG: validate( {"provider":{"metadata":{"name":"WeChat","category":"Social media","message":"Make sure this WeChat key gets revoked, changed, and removed from the code."},"rules":[{"id":"wechat-app-key","rspecKey":"S6652","metadata":{"name":"WeChat app keys should not be disclosed"},"detection":{"matching":{"pattern":"\\bwx[a-f0-9]{16}\\b(?:[^\\r\\n]*?\\r?\\n){0,5}?(?:[^\\r\\n]*?)\\b([a-f0-9]{32})\\b"},"post":{"patternNot":"0{5}|123456","statisticalFilter":{"threshold":3.0}}},"examples":[{"text":"@Schema(description = \"公众号 appId\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"wx8b3a83d0f4efa807\")\n@NotEmpty(message = \"公众号 appId 不能为空\")\nprivate String appId;\n\n@Schema(description = \"公众号密钥\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"40b6b70508b47cbfb4ee39feb617a05a\")\n@NotEmpty(message = \"公众号密钥不能为空\")\n","containsSecret":true,"match":"40b6b70508b47cbfb4ee39feb617a05a"},{"text":"var appId = \"wxdc4e0888857e858d\";\nvar nonceStr = \"2858338391442441211798337075320\";\nvar package = \"prepay_id=wx20171214105006b93f3bb9e90854903185\";\nvar paySign = \"BAF449D858CFD0CE6D488146DEF41A64\";\nvar timeStamp = \"1513219805815\";\n\nwindow.onload = function onBridgeReady(){\n\n    wx.requestPayment({\n        'timeStamp': \"1395712654\",\n        'nonceStr': \"e61463f8efa94090b1f366cccfbbb444\",\n        'package': \"prepay_id=u802345jgfjsdfgsdg888\",\n","containsSecret":false}]}]}}, {"provider":{"metadata":{"name":"WeChat","category":"Social media","message":"Make sure this WeChat key gets revoked, changed, and removed from the code."},"rules":[{"id":"wechat-app-key","rspecKey":"S6652","metadata":{"name":"WeChat app keys should not be disclosed"},"detection":{"matching":{"pattern":"\\bwx[a-f0-9]{16}\\b(?:[^\\r\\n]*?\\r?\\n){0,5}?(?:[^\\r\\n]*?)\\b([a-f0-9]{32})\\b"},"post":{"patternNot":"0{5}|123456","statisticalFilter":{"threshold":3.0}}},"examples":[{"text":"@Schema(description = \"公众号 appId\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"wx8b3a83d0f4efa807\")\n@NotEmpty(message = \"公众号 appId 不能为空\")\nprivate String appId;\n\n@Schema(description = \"公众号密钥\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"40b6b70508b47cbfb4ee39feb617a05a\")\n@NotEmpty(message = \"公众号密钥不能为空\")\n","containsSecret":true,"match":"40b6b70508b47cbfb4ee39feb617a05a"},{"text":"var appId = \"wxdc4e0888857e858d\";\nvar nonceStr = \"2858338391442441211798337075320\";\nvar package = \"prepay_id=wx20171214105006b93f3bb9e90854903185\";\nvar paySign = \"BAF449D858CFD0CE6D488146DEF41A64\";\nvar timeStamp = \"1513219805815\";\n\nwindow.onload = function onBridgeReady(){\n\n    wx.requestPayment({\n        'timeStamp': \"1395712654\",\n        'nonceStr': \"e61463f8efa94090b1f366cccfbbb444\",\n        'package': \"prepay_id=u802345jgfjsdfgsdg888\",\n","containsSecret":false}]}]}}, $)
2023-08-09T21:40:33.8036053Z 21:40:33.406 DEBUG: validate( {"metadata":{"name":"WeChat","category":"Social media","message":"Make sure this WeChat key gets revoked, changed, and removed from the code."},"rules":[{"id":"wechat-app-key","rspecKey":"S6652","metadata":{"name":"WeChat app keys should not be disclosed"},"detection":{"matching":{"pattern":"\\bwx[a-f0-9]{16}\\b(?:[^\\r\\n]*?\\r?\\n){0,5}?(?:[^\\r\\n]*?)\\b([a-f0-9]{32})\\b"},"post":{"patternNot":"0{5}|123456","statisticalFilter":{"threshold":3.0}}},"examples":[{"text":"@Schema(description = \"公众号 appId\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"wx8b3a83d0f4efa807\")\n@NotEmpty(message = \"公众号 appId 不能为空\")\nprivate String appId;\n\n@Schema(description = \"公众号密钥\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"40b6b70508b47cbfb4ee39feb617a05a\")\n@NotEmpty(message = \"公众号密钥不能为空\")\n","containsSecret":true,"match":"40b6b70508b47cbfb4ee39feb617a05a"},{"text":"var appId = \"wxdc4e0888857e858d\";\nvar nonceStr = \"2858338391442441211798337075320\";\nvar package = \"prepay_id=wx20171214105006b93f3bb9e90854903185\";\nvar paySign = \"BAF449D858CFD0CE6D488146DEF41A64\";\nvar timeStamp = \"1513219805815\";\n\nwindow.onload = function onBridgeReady(){\n\n    wx.requestPayment({\n        'timeStamp': \"1395712654\",\n        'nonceStr': \"e61463f8efa94090b1f366cccfbbb444\",\n        'package': \"prepay_id=u802345jgfjsdfgsdg888\",\n","containsSecret":false}]}]}, {"provider":{"metadata":{"name":"WeChat","category":"Social media","message":"Make sure this WeChat key gets revoked, changed, and removed from the code."},"rules":[{"id":"wechat-app-key","rspecKey":"S6652","metadata":{"name":"WeChat app keys should not be disclosed"},"detection":{"matching":{"pattern":"\\bwx[a-f0-9]{16}\\b(?:[^\\r\\n]*?\\r?\\n){0,5}?(?:[^\\r\\n]*?)\\b([a-f0-9]{32})\\b"},"post":{"patternNot":"0{5}|123456","statisticalFilter":{"threshold":3.0}}},"examples":[{"text":"@Schema(description = \"公众号 appId\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"wx8b3a83d0f4efa807\")\n@NotEmpty(message = \"公众号 appId 不能为空\")\nprivate String appId;\n\n@Schema(description = \"公众号密钥\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"40b6b70508b47cbfb4ee39feb617a05a\")\n@NotEmpty(message = \"公众号密钥不能为空\")\n","containsSecret":true,"match":"40b6b70508b47cbfb4ee39feb617a05a"},{"text":"var appId = \"wxdc4e0888857e858d\";\nvar nonceStr = \"2858338391442441211798337075320\";\nvar package = \"prepay_id=wx20171214105006b93f3bb9e90854903185\";\nvar paySign = \"BAF449D858CFD0CE6D488146DEF41A64\";\nvar timeStamp = \"1513219805815\";\n\nwindow.onload = function onBridgeReady(){\n\n    wx.requestPayment({\n        'timeStamp': \"1395712654\",\n        'nonceStr': \"e61463f8efa94090b1f366cccfbbb444\",\n        'package': \"prepay_id=u802345jgfjsdfgsdg888\",\n","containsSecret":false}]}]}}, $.provider)
2023-08-09T21:40:33.8045645Z 21:40:33.407 DEBUG: validate( [{"id":"wechat-app-key","rspecKey":"S6652","metadata":{"name":"WeChat app keys should not be disclosed"},"detection":{"matching":{"pattern":"\\bwx[a-f0-9]{16}\\b(?:[^\\r\\n]*?\\r?\\n){0,5}?(?:[^\\r\\n]*?)\\b([a-f0-9]{32})\\b"},"post":{"patternNot":"0{5}|123456","statisticalFilter":{"threshold":3.0}}},"examples":[{"text":"@Schema(description = \"公众号 appId\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"wx8b3a83d0f4efa807\")\n@NotEmpty(message = \"公众号 appId 不能为空\")\nprivate String appId;\n\n@Schema(description = \"公众号密钥\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"40b6b70508b47cbfb4ee39feb617a05a\")\n@NotEmpty(message = \"公众号密钥不能为空\")\n","containsSecret":true,"match":"40b6b70508b47cbfb4ee39feb617a05a"},{"text":"var appId = \"wxdc4e0888857e858d\";\nvar nonceStr = \"2858338391442441211798337075320\";\nvar package = \"prepay_id=wx20171214105006b93f3bb9e90854903185\";\nvar paySign = \"BAF449D858CFD0CE6D488146DEF41A64\";\nvar timeStamp = \"1513219805815\";\n\nwindow.onload = function onBridgeReady(){\n\n    wx.requestPayment({\n        'timeStamp': \"1395712654\",\n        'nonceStr': \"e61463f8efa94090b1f366cccfbbb444\",\n        'package': \"prepay_id=u802345jgfjsdfgsdg888\",\n","containsSecret":false}]}], {"provider":{"metadata":{"name":"WeChat","category":"Social media","message":"Make sure this WeChat key gets revoked, changed, and removed from the code."},"rules":[{"id":"wechat-app-key","rspecKey":"S6652","metadata":{"name":"WeChat app keys should not be disclosed"},"detection":{"matching":{"pattern":"\\bwx[a-f0-9]{16}\\b(?:[^\\r\\n]*?\\r?\\n){0,5}?(?:[^\\r\\n]*?)\\b([a-f0-9]{32})\\b"},"post":{"patternNot":"0{5}|123456","statisticalFilter":{"threshold":3.0}}},"examples":[{"text":"@Schema(description = \"公众号 appId\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"wx8b3a83d0f4efa807\")\n@NotEmpty(message = \"公众号 appId 不能为空\")\nprivate String appId;\n\n@Schema(description = \"公众号密钥\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"40b6b70508b47cbfb4ee39feb617a05a\")\n@NotEmpty(message = \"公众号密钥不能为空\")\n","containsSecret":true,"match":"40b6b70508b47cbfb4ee39feb617a05a"},{"text":"var appId = \"wxdc4e0888857e858d\";\nvar nonceStr = \"2858338391442441211798337075320\";\nvar package = \"prepay_id=wx20171214105006b93f3bb9e90854903185\";\nvar paySign = \"BAF449D858CFD0CE6D488146DEF41A64\";\nvar timeStamp = \"1513219805815\";\n\nwindow.onload = function onBridgeReady(){\n\n    wx.requestPayment({\n        'timeStamp': \"1395712654\",\n        'nonceStr': \"e61463f8efa94090b1f366cccfbbb444\",\n        'package': \"prepay_id=u802345jgfjsdfgsdg888\",\n","containsSecret":false}]}]}}, $.provider.rules)
2023-08-09T21:40:33.8053864Z 21:40:33.409 DEBUG: validate( {"id":"wechat-app-key","rspecKey":"S6652","metadata":{"name":"WeChat app keys should not be disclosed"},"detection":{"matching":{"pattern":"\\bwx[a-f0-9]{16}\\b(?:[^\\r\\n]*?\\r?\\n){0,5}?(?:[^\\r\\n]*?)\\b([a-f0-9]{32})\\b"},"post":{"patternNot":"0{5}|123456","statisticalFilter":{"threshold":3.0}}},"examples":[{"text":"@Schema(description = \"公众号 appId\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"wx8b3a83d0f4efa807\")\n@NotEmpty(message = \"公众号 appId 不能为空\")\nprivate String appId;\n\n@Schema(description = \"公众号密钥\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"40b6b70508b47cbfb4ee39feb617a05a\")\n@NotEmpty(message = \"公众号密钥不能为空\")\n","containsSecret":true,"match":"40b6b70508b47cbfb4ee39feb617a05a"},{"text":"var appId = \"wxdc4e0888857e858d\";\nvar nonceStr = \"2858338391442441211798337075320\";\nvar package = \"prepay_id=wx20171214105006b93f3bb9e90854903185\";\nvar paySign = \"BAF449D858CFD0CE6D488146DEF41A64\";\nvar timeStamp = \"1513219805815\";\n\nwindow.onload = function onBridgeReady(){\n\n    wx.requestPayment({\n        'timeStamp': \"1395712654\",\n        'nonceStr': \"e61463f8efa94090b1f366cccfbbb444\",\n        'package': \"prepay_id=u802345jgfjsdfgsdg888\",\n","containsSecret":false}]}, {"provider":{"metadata":{"name":"WeChat","category":"Social media","message":"Make sure this WeChat key gets revoked, changed, and removed from the code."},"rules":[{"id":"wechat-app-key","rspecKey":"S6652","metadata":{"name":"WeChat app keys should not be disclosed"},"detection":{"matching":{"pattern":"\\bwx[a-f0-9]{16}\\b(?:[^\\r\\n]*?\\r?\\n){0,5}?(?:[^\\r\\n]*?)\\b([a-f0-9]{32})\\b"},"post":{"patternNot":"0{5}|123456","statisticalFilter":{"threshold":3.0}}},"examples":[{"text":"@Schema(description = \"公众号 appId\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"wx8b3a83d0f4efa807\")\n@NotEmpty(message = \"公众号 appId 不能为空\")\nprivate String appId;\n\n@Schema(description = \"公众号密钥\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"40b6b70508b47cbfb4ee39feb617a05a\")\n@NotEmpty(message = \"公众号密钥不能为空\")\n","containsSecret":true,"match":"40b6b70508b47cbfb4ee39feb617a05a"},{"text":"var appId = \"wxdc4e0888857e858d\";\nvar nonceStr = \"2858338391442441211798337075320\";\nvar package = \"prepay_id=wx20171214105006b93f3bb9e90854903185\";\nvar paySign = \"BAF449D858CFD0CE6D488146DEF41A64\";\nvar timeStamp = \"1513219805815\";\n\nwindow.onload = function onBridgeReady(){\n\n    wx.requestPayment({\n        'timeStamp': \"1395712654\",\n        'nonceStr': \"e61463f8efa94090b1f366cccfbbb444\",\n        'package': \"prepay_id=u802345jgfjsdfgsdg888\",\n","containsSecret":false}]}]}}, $.provider.rules[0])
2023-08-09T21:40:33.8061726Z 21:40:33.410 DEBUG: validate( [{"text":"@Schema(description = \"公众号 appId\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"wx8b3a83d0f4efa807\")\n@NotEmpty(message = \"公众号 appId 不能为空\")\nprivate String appId;\n\n@Schema(description = \"公众号密钥\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"40b6b70508b47cbfb4ee39feb617a05a\")\n@NotEmpty(message = \"公众号密钥不能为空\")\n","containsSecret":true,"match":"40b6b70508b47cbfb4ee39feb617a05a"},{"text":"var appId = \"wxdc4e0888857e858d\";\nvar nonceStr = \"2858338391442441211798337075320\";\nvar package = \"prepay_id=wx20171214105006b93f3bb9e90854903185\";\nvar paySign = \"BAF449D858CFD0CE6D488146DEF41A64\";\nvar timeStamp = \"1513219805815\";\n\nwindow.onload = function onBridgeReady(){\n\n    wx.requestPayment({\n        'timeStamp': \"1395712654\",\n        'nonceStr': \"e61463f8efa94090b1f366cccfbbb444\",\n        'package': \"prepay_id=u802345jgfjsdfgsdg888\",\n","containsSecret":false}], {"provider":{"metadata":{"name":"WeChat","category":"Social media","message":"Make sure this WeChat key gets revoked, changed, and removed from the code."},"rules":[{"id":"wechat-app-key","rspecKey":"S6652","metadata":{"name":"WeChat app keys should not be disclosed"},"detection":{"matching":{"pattern":"\\bwx[a-f0-9]{16}\\b(?:[^\\r\\n]*?\\r?\\n){0,5}?(?:[^\\r\\n]*?)\\b([a-f0-9]{32})\\b"},"post":{"patternNot":"0{5}|123456","statisticalFilter":{"threshold":3.0}}},"examples":[{"text":"@Schema(description = \"公众号 appId\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"wx8b3a83d0f4efa807\")\n@NotEmpty(message = \"公众号 appId 不能为空\")\nprivate String appId;\n\n@Schema(description = \"公众号密钥\", requiredMode = Schema.RequiredMode.REQUIRED, example = \"40b6b70508b47cbfb4ee39feb617a05a\")\n@NotEmpty(message = \"公众号密钥不能为空\")\n","containsSecret":true,"match":"40b6b70508b47cbfb4ee39feb617a05a"},{"text":"var appId = \"wxdc4e0888857e858d\";\nvar nonceStr = \"2858338391442441211798337075320\";\nvar package = \"prepay_id=wx20171214105006b93f3bb9e90854903185\";\nvar paySign = \"BAF449D858CFD0CE6D488146DEF41A64\";\nvar timeStamp = \"1513219805815\";\n\nwindow.onload = function onBridgeReady(){\n\n    wx.requestPayment({\n        'timeStamp': \"1395712654\",\n        'nonceStr': \"e61463f8efa94090b1f366cccfbbb444\",\n        'package': \"prepay_id=u802345jgfjsdfgsdg888\",\n","containsSecret":false}]}]}}, $.provider.rules[0].examples)
5

Hi,

Thanks for the followup. We rolled out changes to the Secrets sensor last week. That sensor looks for possible hard-coded secrets, and it looks like it got a little chatty in the debug logs. I’ll bring that up with the relevant team.

 
Thx,
Ann

1 Like
8

Hi @Saqwel ,

thanks for the report!
You’re right. Apparently one of our included libraries will pollute the log during scanning in debug mode, which should not be the case.

Out of curiosity, is there a specific need to run the sensor in the debug mode with sonar.verbose = true?

To solve this, I created a ticket that will address the issue in the upcoming release of the secrets sensor.

Best,
Jonas

9

No there is no any specific need to run anaysis in debug mode. That was done just in case.
Thank you for creating ticket.

12

We got caught out by this too. Setting sonar.verbose=false resolved the issue. Thanks for posting.

1 Like