Parallel processing within a project is available but off by default.
Note that even once you enable it, there are still some limits on parallel processing branches from the same project (what if one thread is processing a main analysis while another one is processing a branch that uses main as reference?).
The option to perform multiple runs in parallel is already enabled. Additionally, none of the builds being run are for the main branch. Currently, one PR build is being run for a branch, and four are queued for other branches on the same project. We have two workers sitting idle and not picking up the queued jobs.
CI-side you can analyze everything in parallel. (You knew this already, but for posterity’s sake…) On the server side, branches from the same project will not be processed in parallel. What will be processed in parallel is PRs from the same project and branches from different projects. So, these can all run at the same time:
Notice, we have one branch from each project and multiple PRs. Now, I doubt many people have enough CE workers to actually run all those^ in parallel, but if all of those things were queued at (approximately) the same time, there would be no idle workers.
Hey Ann!
I work with Chris and the main issue that caused us to look into this is we are getting long queue times for scans. Its all for one project with many branches that are backing up as the other projects are able to take the other workers no problem. I am wondering if you have any ideas on how we could speed up our scans?
Yeah, we ran that right after the upgrade and went from a 24 hour queue time to averaging 1-8 hours.
Today this project has a 4 hour wait time on builds, which is way too long. Does SQ itself collect any performance metrics on the CEs to help troubleshoot?
I’m working on aggregating metrics across projects to spot trends in build times since the upgrade, but anecdotally it looks like some of our build times are up more than 50%.
Probably your /logs/ce.log file will clue you into where time is being spent during the CE processing. If you can narrow it down to the specific project that is taking a while, extract those logs and say, run a grep to get all time=123ms with 6 digits
grep -E 'time=[0-9]{6,}ms' ce.log
Ultimately you might fiddle with this filter (swap out 6 for 5 or 7) based on what your find.