Subscribe
The health of your software development life cycle (SDLC) is an important indicator of your organizations’ quality assurance, cost effectiveness, customer satisfaction, and compliance. While the executive order (EO) on improving the nation's cybersecurity issued in May 2021 only required software Bill of Materials (SBOM)s for federal agencies, it has resulted in widespread promotion and adoption of SBOMs as an industry best practice. The EO has also driven many organizations to re-evaluate their software development processes and implement measures to improve the security and resilience of their software supply chains. It also underscores that organizations need to pay closer attention to their development practices and health, as SBOM creation can expose “dirty laundry” to the world. This is why it’s time to stop using pen tests to simply find problems in your software, and start using them to find problems in your processes.
Pen testing as SDLC best practice
While software developers have long used third-party web app and API pen tests to find application security defects, pen tests are also a great way to gauge the health of an SDLC. Third-party automated pen tests, which are often mandated by regulatory agencies, require very little experience with tooling or training on the part of the customer, and are fairly thorough in identifying security flaws that need to be fixed. This is why they’re popular. From both parties' points of view, it's a pretty fulfilling activity; the pen tester gets to hand over a number of alarming-sounding vulnerabilities, and the developers receive a number of defects to add to the backlog.
However, what third-party automated pen tests cannot replace is the human behind pen testing execution. While humans are slow and more expensive than automated defect discovery tooling, because they can mimic human hackers, they are better at evaluating an application's response to a pen test, and can potentially catch responses that automated software may miss.
One way to think about how you are using pen tests is to look at the cost of finding defects. For example, finding a cross-site scripting defect using a SAST tool is much more efficient than finding it using a pen test. Using pen tests for vulnerability detection means that each one costs a percentage of your limited testing time, a fraction of your checklist, and a portion of your report writeup effort. And when companies run pen tests on an annual basis, the cost per defect increases as vulnerabilities get patched, and the critical and high vulnerabilities dwindle to nothing. So if pen testing shows diminishing returns as a defect discovery tool, how can you use the expensive but more accurate human element of pen tests to take full advantage of their value?
Where pen testing really begins to pay off is when organizations leave routine defect discovery to automated tools and shift the human effort toward AppSec program assurance. Just as there are multiple points in a piece of software that could validate, detect, or encode a cross-site scripting attack, there are multiple points in an SDLC that can anticipate, prevent, detect, or mitigate various classes of vulnerabilities. By shifting pen testing efforts to finding those inflection points in your SDLC where vulnerabilities can be prevented, you’re using the human and financial costs of pen testing more efficiently than if you’re just focused on defect discovery. Using pen testing this way can help you detect the processes in your SDLC that allow vulnerabilities to creep in, so if you begin fixing those processes, you’ll also minimize future vulnerabilities.
Rather than relying on pen tests to detect security flaws that must be patched individually, pen testing should be used to perform a blameless postmortem, and analyze whether improvements are needed to ensure that potential failures are recognized at specific points in the SDLC. There are many defect identification systems that can find single types of issues, but they are usually not great at finding edge cases. For that, you need human beings, and this is where the pen test can really work for your organization.
Using pen test results to secure your SDLC and your code
You can use pen test results in various ways to secure your SDLC as well as your code.
- Policy and standards: Update your policy and standards to explicitly state that production defects are not tolerated, and fund efforts for training, tooling, and prioritizing defect detection and remediation. Senior leadership can approve and enforce policy and standard changes based on pen test results.
- Requirements: Start writing reusable, implementable, and testable security requirements that address the OWASP Top 10 vulnerabilities. When pen tests detect security defects, write user stories and approval criteria to prevent those defects .
- Training: When your pen tests detect a security fault , determine whether your training's learning objectives cover this issue. How can developers achieve security standards without security fault prevention training? If developers aren't trained to prevent security flaws, the training program has failed, not the development team.
- Threat modeling and design review: Proactively finding and preventing whole classes of problems can be done by looking at the design. Use your pen testing results to modify threat modeling checklists to cover design decisions that could prevent or mitigate security faults, and to establish secure design patterns that allow developers to rule out entire classes of defects.
- Instead of relying on pen testing, shift to in-band SDLC flaw discovery tooling to discover and remediate entire classes of flaws.
- Use static AST or code review checklists to prevent industry-known vulnerabilities that naïve developers miss. While reviewing pen test findings, examine whether you can use static defect discovery capabilities to find these issues and tune your testing to ensure they are found.
- You can use dynamic application security testing (DAST) or interactive application security testing (IAST) to discover runtime bugs and setup concerns that pen tests typically disclose.
- Instead of using pen tests, you can use QA-based security tests to uncover edge cases and logic flaws that scanning rule sets miss. When architecture design review and threat modeling identify dangerous logic, abuse scenarios should be created and run by the automated pipeline.
- Instead of pen testing after the fact, your production scans should practice like they play. By having the testing environment as close to the deployment environment as possible in terms of configuration settings, rights, and infrastructure, vulnerabilities caused by configuration settings can be discovered and resolved in the gold image before it is deployed to production.
- After pen testing detects flaws, you need to build gating mechanisms to catch and stop them from moving down the SDLC. After determining a defect's requirement, misuse cases, implementation guidelines, and testing technique, add checks to the life cycle gates to catch and prevent them from going into production.
- Production risk management should check for and mitigate production defects because it's never too late to do things right. Using your pen testing results you can automate detection and send changes back for rework, or you can implement manual push-to-release checks that reduce environmental risk.
Once these vulnerability issues are covered, you can begin to reposition pen testing in the risk management portfolio. When delivering penetration test results, there should be tooling and knowledge in place to facilitate a discussion with teams to not only talk about how to fix it in the code, but with the AppSec program owners about which of their capabilities may not be operating at the level they think they are. When incorporating pen test results into your SDLC, the results can help prioritize recommendations, but pen testing can’t provide direct evidence for training needs, compliance enforcement, vulnerability management needs, security champions, or other key capabilities that don’t directly prevent or detect pen test security defects.
It's time to rethink how pen tests are used in risk management. There are cheaper ways to find defects and faster ways to reduce risk in an application, but the pen test’s human-driven nature means that it will thoroughly exercise whichever capabilities are not being capitalized on.
Continue Reading
