Extra accreditation and compliance necessities have been added in response to cyber incidents. Whereas these frameworks play an vital position in establishing security baselines, true security is extra than simply reaching an ideal compliance rating. As I typically say, “insurance policies and procedures gained’t cease an attacker, they’ll simply have extra paperwork to exfiltrate after they breach us.”
Testing how our environments face up to a decided menace actor is the actual validation of security posture. That’s the place the annual guide penetration check is available in, with boards now demanding to see optimistic outcomes.
There are, nevertheless, important points with guide penetration testing I’ve skilled, notably when carried out solely yearly.
Pace, scope, and the human bottleneck
The constraints of guide testing turned more and more obvious as the environment grew extra complicated. Each engagement was certain by time and funds, forcing troublesome trade-offs about what to check and the way deeply. The standard and comprehensiveness of outcomes various considerably relying on which advisor we engaged, their particular person experience, their familiarity with rising methods, and the way a lot they might accomplish throughout the contracted hours.
Conventional penetration testing delivered what I got here to see as a essentially flawed worth proposition. We’d make investments important funds to obtain a snapshot of our security posture weeks after the check concluded and from that second it started getting older like milk. There was no ongoing suggestions loop, no steady validation of our security controls. We had been primarily flying blind between annual checks, hoping our defenses remained efficient even because the menace panorama developed day by day round us.
The remediation black gap
Maybe most irritating was what occurred after we obtained findings. Our groups would work diligently to implement fixes, however we hardly ever had the funds or alternative to carry testers again to validate remediation. We had been left with uncertainty. This hole between identification and verification created a harmful blind spot in our security program.
Conventional vulnerability assessments leaned closely on CVSS severity scores that didn’t inform us how exploitable a vulnerability was in our particular surroundings or the place it sat inside a practical assault path. We would have liked to grasp what an attacker might really accomplish by chaining vulnerabilities collectively.
A greater manner ahead
Pissed off with these limitations, I explored automated penetration testing, a class that features breach and assault simulation (BAS) and steady automated purple teaming (CART). Platforms like Pentera and Horizon3.ai’s NodeZero conduct steady, on-demand simulations utilizing real-world attacker techniques, methods, and procedures.
They provide black field testing (simulating exterior attackers), gray field testing (simulating insider threats), and customized situations focusing on particular dangers like ransomware or zero-day exploits.
Most significantly, they ship outcomes immediately, no ready weeks for studies, and allow rapid retesting to validate fixes.
The implementation and funding
We moved from $35,000 for an annual guide check to $90,000 yearly for an automatic platform, delivering over $1.3 million value of equal testing. Our cadence jumped from one check per yr to a minimal of 38, with limitless flexibility for added simulations.
We established a fortnightly rhythm of black field and gray field checks, supplemented by month-to-month customized situations focusing on particular considerations like ransomware assaults. This gave our workforce two weeks to remediate earlier than retesting confirmed fixes labored. These instruments check extra in a day than human testers accomplish in every week, quickly adjusting to findings and leveraging gaps to probe deeper.
Sudden classes and workforce transformation
The platform delivered insights that essentially modified our understanding. Take password security: we’d adopted longer passphrases, assured that fourteen-character phrases would improve breach time from eight months to 12 billion years. The software shattered that confidence, cracking a 23-character passphrase containing upper- and lower-case letters, numbers, and particular characters in underneath half an hour. The lesson was humbling, people are predictable. Attackers keep wordlists and precomputed hash lists in rainbow tables particularly focusing on widespread phrases. Passphrase size issues, however high quality issues extra.
The retesting capabilities proved sport altering. Safety groups might determine issues, remediate them, and instantly retest to confirm fixes had been efficient. The platform generated each executive-level studies for board displays and detailed technical studies for security groups to motion immediately, not weeks later.
Maybe most significantly, the platform elevated our workforce’s functionality. Till your workforce experiences an automatic penetration testing software exploiting their surroundings, they gained’t absolutely comprehend the way to apply defensive ideas to their particular programs. Every simulated assault was absolutely documented, offering real-time studying alternatives. The groups started treating the platform as a sport they had been decided to win.
Rethinking prioritization: assault paths over severity scores
Probably the most important revelations was how automated penetration testing reworked our vulnerability administration. We found that the critical-rated vulnerability receiving rapid consideration could be buried 5 layers deep in an assault path, whereas a low-rated vulnerability we’d deprioritized could possibly be the preliminary entry level attackers would exploit. Extra revealing nonetheless, the platform confirmed how seemingly low-risk vulnerabilities could possibly be chained collectively to entry crucial programs.
This modified our patching technique. As an alternative of reflexively addressing vulnerabilities by CVSS severity rankings, we targeted on what attackers might really use to ascertain a foothold. Given the overwhelming variety of vulnerabilities requiring fixed consideration, this intelligence about precise assault pathways proved invaluable permitting us to focus restricted assets the place they’d produce the best security final result somewhat than chasing severity scores that didn’t replicate real-world danger.
The hole between configuration and actuality
We place monumental religion in our security tooling after we allow a characteristic, we assume it’s working. The automated penetration testing platform delivered a sobering lesson: check your controls, don’t simply belief the GUI.
I skilled this firsthand after we enabled a performance to mitigate a particular danger. It regarded good on display, however it wasn’t working. The platform methodically examined totally different assault varieties, together with the state of affairs we thought we’d protected in opposition to. The assault succeeded, the security software’s options weren’t functioning as a consequence of a bug. We didn’t have the safety we thought we did.
It jogs my memory of the defender’s dilemma: “Defenders must be proper 100% of the time; attackers solely must get it proper as soon as.” I’d a lot choose our personal testing instruments spotlight these gaps than have attackers uncover them.
The final word validation: Testing your detection and response
One other highly effective software is validating your detection instruments and SOC. The primary time I ran a proof of idea, I intentionally didn’t inform our third-party SOC. Our inner SIEM instantly generated quite a few alerts. It took 4 hours for the exterior SOC to contact us — a lifetime in cybersecurity.
If you’re paying for a third-party service, validating their response is invaluable and I strongly suggest working at the very least one unannounced check. The outcomes might shock you, and it’s much better to find gaps throughout your personal testing than throughout an precise incident.
One last lesson: as your security resilience improves and also you obtain persistently excessive scores, you attain a plateau. Shifting to a brand new automated penetration testing platform can yield recent findings, as every software takes totally different approaches, offering alternatives to proceed enhancing somewhat than changing into complacent.
The decision: Evolution, not elimination
Must you substitute guide penetration testing with automated platforms? The reply is nuanced. For ongoing security validation, steady enchancment, and operational resilience, automated testing ought to turn into your main validation technique. The ROI, studying alternatives, and steady suggestions loop far exceed what annual guide testing delivers.
Nonetheless, I wouldn’t fully eradicate guide testing. There’s nonetheless worth in bringing in specialised human testers for complicated customized functions, crucial infrastructure adjustments, or while you want inventive pondering that solely skilled security researchers present. Consider automated platforms as your day by day coaching routine, with guide checks as occasional specialised assessments.
The true query is whether or not you’ll be able to afford not to undertake steady automated validation. The hole between annual guide checks leaves you susceptible for 364 days a yr. Automated penetration testing fills that hole, transforms your workforce’s capabilities, and validates your security posture repeatedly, not simply yearly when auditors ask.
