Hi all,
Thank you to everyone who attended our webinar! You will find below the questions that were asked during the session, as well as detailed answers and resources.
Q: Is that traditional cost model fit for software development? It was developed for different industries that involve manual labor.
Jonathan Slaughter: Great question. tl;dr is Yes.
While most quality models vastly predate modern software development practices, their foundations are all based on the same principles: delivering the desired outcome effectively, efficiently, and sustainably…
These models also support making connections between the software development world, and the other parts of an organization that are often reliant on the technology, or have an expectation to assess it.
If you are looking for an excellent, software engineering-focused quality model, take a look at ISO 25010:2023, SQuaRE Product Quality, which I mentioned in the live Q&A session.
It breaks down product quality into eight (8) product quality characteristics and 31 sub-characteristics. While I won’t share them all, the 8 main characteristics are: Functionality Suitability, Reliability, Performance Efficiency, Usability, Security, Compatibility, Maintainability, and Portability.
These 8 actually are subparts of the three (3) points that define “Good” Quality from the presentation:
- Functionality, Suitability, Reliability (Effectiveness)
- Performance Efficiency, Usability (Efficiency)
- Security, Compatibility, Maintainability, Portability (Appropriate for Current and Future Use)
Thanks again for taking the time to attend the webinar, and for the question.
Q: According to GPT the movie should be “The Producers”
Jonathan Slaughter: Far be it from me to disagree with ChatCPT on anything. However, the specific reference point was from “Spaceballs.”
In the scene where Lone Starr and crew are leaving Yogurt’s place and Yogurt gives him the ring, Lone Starr asks, “I wonder, will we ever see each other again?” Yogurt then replies, “Who knows? God willing, we’ll all meet again in ‘Spaceballs 2: The Search for More Money.’”
Q: Are there any references for the costs cited in the PAF(F) part?
Jonathan Slaughter: So, I am a big fan of the Chartered Quality Institute (CQI) and the International Register of Certified Auditors (IRCA). They have a great website, www.quality.org, which is filled with excellent resources.
Specifically for digital/software/technology and PAF(F), I would say start with this excellent article from there, Quantifying quality costs in the digital age | CQI | IRCA, and after you read it, download the PDF at the bottom titled “The True Cost of Quality”.
The presentation is fantastic, and also I feel that the material is a logical step deeper in the discussion, without forcing you down a single way of thinking.
If you want to dig even deeper, I would recommend an older book called Poor-Quality Costs by Dr. H. James Harrington. Someone asked about favorite books on quality, and this is definitely one of them. I don’t have it in my list below because it was written in 1987, and like many things released that year (Cool Ranch Doritos, Spaghetti-Os, T’Pau’s “Heart and Soul”) it can be hard to find, or seen as outdated. And as with Cool Ranch Doritos and Spaghetti-Os, the book is still amazing for the purpose it serves. 
Q: Is there proof for the factor of 10 statements? I’ve seen these pitches before but I have found it very hard to find any evidence for this.
Jonathan Slaughter: Short answer is “Yes.” The longer answer is more nuanced.
As I mentioned in the Q&A session at the end of the webinar, the Factor of 10 will have some deviation from organization and industry type. However, I have done this analysis at multiple companies and the results are at, or near, factor of 10 each times.
The issue for most companies is that they have not actually laid out their current costs in the right order, as according to PAF(F) modeling.
For example, Security operations (SOC) staff are often listed as Preventive. However, that is not the case. Security staff and tooling are more internal failures because those resources are dedicated to catching errors after they happen.
Even the training of this staff - yes it is “preventive” to make sure that those security individuals have the right training to do their job, but their job and focus is not directly related to prevention within the larger design and development of the product.
However, a Security GRC resource whose job is focused on supporting the governance for the tool within the product and process design and provisioning training to the front-line developer personnel is more aligned with actual Prevention costs.
In answer to the next question below, I have provided a breakdown of the cost types for each stage of PAF(F), which is a good starting point, and I encourage you to use a 5-Why’s, or even a 7 Wastes Analysis model to identify where the costs you identify truly sit, and their overall efficiency and effectiveness.
If there is a need to chat further, happy to provide further guidance, as it makes sense.
Q: Is there any suggestion for finding vulnerabilities in non-compilable code? Or code snippets?
A: You might be interested in Automatic analysis, only available with SonarCloud and GitHub at the moment: Automatic analysis & SonarCloud
Q: Can Sonar Scanner perform type checks and security checks for Python code?
A: Yes, it can do both: for type check, see here: Python static code analysis.
For security checks, see here: Python static code analysis | Vulnerability
Q: What are some tangible ways of utilizing software code “Quality” in driving or positively influencing software code “Security”. i.e. minimize “security” role & required efforts through well-engineered software “Quality” measures?
Jonathan Slaughter: So, I am torn on the best way to answer this question. There are so many ways this could go, and each organization is different. I am doing my best to be more like Google PageRank than AltaVista/TCS Rolodex with my answers, so I will go with what I think is the most relevant and refer you to ISO 25010:2023. This is an excellent software standard and is focused on the SQuaRE model, with appropriate references to reasonable security for achieving “Good” quality outcomes.
For the AltaVista/TCS Rolodex version of my answers, please feel free to book time in my brain, but pack a lunch.
Q: Is there a mapping cheatsheet between "software code “Quality” and software code “Security” controls/metrics/requirements that can aid a developer at the Prevention/Appraisal phases?
Jonathan Slaughter: This is very similar to what I answered above, and asked by the same individual, so I will just defer to the response to Question 5 and hope this supports both. If not, please let us know.
Q: How would you approach improving a large legacy codebase that was not built with long-term quality in mind from the start?
A: I suggest you read about our methodology called Clean as you Code. The idea is to prevent issues from being added to the codebase, and to improve the historical codebase progressively Clean as You Code & SonarQube
Q: Can SonarQube fetch the code coverage percentage of SalesForce?
A: Yes, I believe it would be possible to do so. See here for more information: Test coverage parameters
Q:Is Apex scanning free in SonarCloud? What is the difference between Sonar Cloud and SonarQube?
A: For public projects (e.g. open-source, or with publicly visible source code), it is. For private projects, not. SonarQube is a self-managed solution that needs to be deployed and operated by you. SonarCloud is a Software as a Service and is therefore managed by us.
Q: This is great content. It’s been said that automation is the enemy of security as, many times, automation sometimes requires opening hooks and broadening the potential attack service in order to allow for more efficient automation control of a product. Can you speak to this apparent contradiction?
Jonathan Slaughter: First of all, thank you so much for the kind words.
Second, I would say this is - at best - only partially correct. I would say that automation and security are more “frenemies” like Wolverine and Deadpool.
The real enemy is complexity. Usually, this is an output of poor-quality design and strategy, where components, dependencies, integrations, and configurations grow at an exponential rate. Each has to be customized and bent to fit into the larger narrative. What starts out as a single product with a simple design often leads to something onerous, and heavy, and requires way too much time and effort to fully grasp the nuances to move forward. To keep with the comic book theme, it is like joining the MCU with the Eternals and wondering where Tony Stark and Steve Rogers are. This is often where opening hooks and broadened attack surfaces really come to bear.
In reality, when you do automation with security embedded in the design, you end up with a truly symbiotic relationship, which leads us to Venom and Eddie Brock - my personal favorite
Q: Would you like to elaborate on similarities and differences between security and safety and quality?
Jonathan Slaughter: Absolutely. To do so, I will treat each as separate, but interrelated outcomes. As outcomes, let’s go with these definitions:
- “Good” Quality:
- The product is effective, efficient, and appropriate for present and anticipated future circumstances
- “Good” Safety:
- The product delivers excepted, or greater, value to the end-user without harming them or the environment
- “Good” Security:
- The measures taken to ensure the safety of the product, user, and other assets (e.g., data) are effective without creating an undue burden on the user’s use of the product.
This is where I will put on the Software-as-a-Medical Device (SaMD) hat, and talk about how each of these (Quality, Safety, Security) are all different risks that need to be assessed in the project.
A common risk for each of these is Integrity.
- For quality management:
- Integrity is around ensuring that the process followed delivers predictable, expected results and outcomes with limited variance and deviation.
- For (cyber)security:
- Integrity means that data or information in your system is not altered or deleted by unauthorized individuals.
If the quality integrity is off on a SaMD, you may get an unexpected outcome in how the device performs, which can lead to HARM of the end user by the operation of the device. Equally, if the security integrity is compromised, the data within the device may be altered, which causes HARM to the user, even if the device provided the correct output based on the (altered) data it received.
If an insulin pump has a quality flaw that misreads or does the wrong process based on correct data, it could deliver a fatal dose of insulin to the patient. The same could happen if the data is altered, and the pump delivers the correct amount of insulin for the altered data, which is too much and ends up being lethal to the user.
This is just one example of how quality, security, and safety are all interrelated.
Q: What are your favorite books about quality?
Jonathan Slaughter: In no particular order, I would say:
- Quality, Productivity, and Competitive Position - W. Edwards Deming
- Measure What Matters - John Doerr
- Juran’s Quality Handbook, 7th Edition - Joseph A. Defeo
I would also say Crossing the Chasm by Geoffrey Moore should be a go-to for everyone, especially if you are really focused on working from outcomes, as opposed to forcing the outcome to squeeze through universal controls
Q: From your experience, what are your recommendations on the best practices to manage security risks?
Jonathan Slaughter: I am a huge fan of inverting. I mentioned it on the webinar about Carl Gustav Jacob Jacobi, a 19th-century mathematician (and the “Father of Elliptical Functions”). He was known for telling his students to “invert” a research topic when studying it, which opened up different ways of studying it.
While not everyone is up to speed in their 19th-century German mathematicians, inverting for security risk management is super-useful.
Start with the outcomes you seek, which are definitions of success for the product, service, etc. Then, instead of asking how to ensure its success, ask how you can ensure its failure.
This is useful for two reasons:
- If forces you to look at things like an attacker, which is useful in security; and
- Ensuring failure is much more in your control than ensuring success - especially in the crowded world of technology.
Once you know how to make it fail, you ask, “What do I do to prevent this specific failure?” and work to implement those protections, controls, etc.
As an added bonus, this works well in security conversations because it creates a common language between security experts and non-security individuals who often are put in the position of using the decided security controls.
A modern practitioner of Inverting was Charlie Munger, who was Warren Buffett’s right-hand man for decades until his recent death at the age of 99. When Warren Buffett listens to you for advice, you are probably worth listening to.
