Security Benchmarks - Calling for your input & suggestions!

Dear Community,

Over the past 3 years we worked hard to bring you a powerful security analysis engine for Java, PHP, C#, Python, JavaScript and TypeScript code. Throughout our development, we’ve analyzed and re-analyzed public benchmarks, intentionally vulnerable apps and past CVEs to iteratively assess and improve its capabilities.

We believe we have very good coverage of common vulnerabilities and we are fine-tuning the engine to make it even more precise (fewer false-positives). And at the same time, we do not want to stop improving :muscle:. Being open and transparent, we’re hoping for your help on this!

My goal with this post is therefore to know if you are aware of any security benchmark (public or private) that could help us in this effort, especially those less known than the established ones already on our radar (OWASP Benchmark, JuiceShop, Juliet Test Suite, OpenSSF, WebGoat, OWASP VulnerableApp, OWASP Mutillidae, … and all the “Damn Vulnerable XYZ”; see the list maintained by OWASP here). We would like to use as many of them as possible to iteratively improve our engine.

Feel free to reply to this thread with your contributions. Or if you prefer you can also DM me on Twitter. We welcome as many references/pointers about each benchmark, so even if your favorite has already been mentioned, it’s valuable to hear from you as well. We’d also be particularly interested to hear your feedback on the benchmarks/apps you’ve used in your own evaluations.

Alex, Code Security PM @ SonarSource