What is penetration testing?

Penetration testing, or "pen testing", is the practice of performing simulated attacks on a network, in order to evaluate the various cyber security controls and processes an organization may deploy to protect IT assets.

Typically performed by an external agent, a penetration test can be performed in a variety of ways, but they all involve attempts to breach an organization's defenses using the same tools that threat actors may deploy.

Despite common wisdom, the main goal of a penetration test is not to simply find vulnerabilities in an organisation's defenses. Instead, a test should be viewed as the primary way of gaining assurance in an organization's vulnerability assessment and management process.

A report from application security vendor Veracode found that automated scans failed to find 62% of Common Weakness Enumeration (CWE) flaws successfully exposed by manual pen testing. It's for this reason that Mordor Intelligence analysis forecasts the testing market will reach $12.8 billion by 2029.

"Some bugs require knowledge of underlying business logic or data flows," Veracode chief research officer Chris Eng points out. "For example, with a page that should only be accessible by certain users, an automated scan generally wouldn’t be able to tell if the appropriate access control check was in place because it doesn’t know the business rules unique to that application."

A penetration tester uses contextual clues to evaluate vulnerability impacts. It can be worse to allow a user to manipulate a price, versus letting them manipulate some other variable - but code-wise these look identical, Eng says.

How penetration tests are deployed

External pen tests may target servers and hardware that any hacker might see, with internal tests typically simulating what happens if hackers cross the network perimeter or an insider threat wants to cause trouble. Either way, a test should never disrupt or put systems at risk.

Tactics strictly reflect organizational context. Conversely, 'straight' vulnerability scanning, while useful, merely inspects points of potential exploitation on the network. Pen testing is more probing and wide-ranging as well as active, rigorous and tailored.

Don't ask testers to simply see what they can find - what comes back may be too costly or overlook business-critical issues. Define structure and goals for testing from the start; know what success looks like so you can judge the results. Is it just about keeping hackers out, or more about vulnerability exploitation and data exposure?

Eng says balance speed against depth. Pen testing can "absolutely" drive a deep dig to unravel issues, but "seeing how far you can get" depends on budget. If 80% of issues are likely identified in week one, is a second testing week worthwhile?

Different organizations will have different thresholds, part-dependent on application criticality and data sensitivities. A good test team will understand how deep they can go within agreed timeframes without sacrificing necessary breadth of coverage, he says.

One evolving approach is a move toward smaller, more frequent penetration tests, with a pool of hours or days used at shorter notice. Instead of testing massively once a year, test new functionality as it’s developed - mirroring faster development timeframes and agile practices.

What should be tested?

"Consider your overall attack surface. If you have a bunch of public-facing applications that are easily exploitable via a bug in some open source library, probably address that before finding and fixing bugs deeper within the system," Eng says.

"I would want to run software composition analysis and possibly an attack surface management (ASM) tool for a big-picture view of lowest hanging fruit before drilling too deeply into any single penetration test."

Certain applications will want pen testing regardless, he adds.

"Even a small 'shop' can be one ransomware attack from being shuttered," agrees Charles Henderson, enterprise vice-president (EVP) of cyber security services at cyber security and compliance services provider Coalfire.

Approaches might include "adversary emulation" - or red teaming. This simulated attack is like a fire drill that might need to be as realistic as possible. It might also be important that internal IT staffers aren't notified of tests prior to completion and reporting, enabling comparisons that highlight the gap between attempted attack and successful defense.

Conversely, the focus may be on how the internal 'blue' team moves from a posture of assumed breach to detect and respond, or a 'purple' mix of teams and approaches, Henderson says.

The critical factor can be how the 'home' team detects and responds to breaches, because that's always possible, he points out.

How to pick a penetration testing company

It's key to analyse teams and skillsets on offer in detail. Security consultants capable of pen testing typically hail from providers who can offer someone with a relevant research background and who can speak authoritatively on the topic.

"If they're relying on the the skills of others on how to test, they're very reactionary," Henderson notes. "You're looking for a firm with really good testers, because no matter how good their methodology is, it comes down to who's on the keyboard."

Even with suitable skillsets, good methodology and enough time and resource are essential. And as usual - if you get pricing that's too good to to be true, it probably is, he says.

Some may be doing mostly tool based testing - although there's absolutely a place for that, he adds, whether in a self-service capacity or as a managed service with oversight.

"We offer it too - but if you're doing penetration testing, you're really looking for that manual use of human beings to connect issues they find and fully understand the gravity of it, rather than just delivering a list of defects across the environment," Henderson explains.

Decide which players you can trust and with what. You might also look at strong recommendations from previous customers and focus on accredited testers, for instance with the global Council of Registered Ethical Security Testers (CREST) certification, which requires examination and adherence to enforceable codes of conduct.

What happens in a penetration test?

As we've noted, the nature of the test depends entirely on what objectives an organization hopes to achieve. 

Typically, though, pen testers perform both external tests, which target the servers and hardware that any hacker would be able to see, and internal tests, which simulate what would happen if those hackers made it past the perimeter and got inside your network, or if an employee wanted to cause trouble. 

Both approaches can be revealing and combined they can provide a good indication of your real-world security position.

An external test may be almost invisible, although, if you have a good security infrastructure, it will hopefully flag up any suspicious connection attempts. An internal test needn't be much more invasive: the tester simply requires access to your network so they can mimic the actions of a hacker.

If that makes you nervous, remember that the testers are looking to expose vulnerabilities, not to exploit them. No data will be compromised, no systems will be interrupted and no damage will be done. Still, it's worth making sure any senior stakeholders have been notified that a pen test is taking place so that they're aware of what's happening.

Tests can take as little as a few hours or last as long as a few weeks, depending on the scope. Just remember that the pen testers' work isn't over when they log out or discontinue their simulated attacks; further time is needed to produce a vulnerability report, after which your business will need time to digest its findings and respond as needed. Indeed, there's a good chance that you will want to involve the testing agency in remedying any issues discovered, so it's worth thinking about keeping them involved for the long-term.

With all tests, you should focus on what needs fixing urgently and be realistic about what's a long-term goal, or what might not be worth fixing at all given the risk appetite of your business. The value of pen testing is that it gives you the information you need to make these decisions.

How to prepare your business for a penetration test

In the interest of simulating an 'attack' that's as realistic as possible, internal IT teams would ideally not be notified of an incoming test before it has been completed and an external report has been compiled. Typically, only a small number of key stakeholders will be aware a test is even happening.

This way, defences will be tested organically, with internal IT teams able to devise their own report accordingly, mirroring the process of an actual attack like a mock fire drill. The two reports can then be compared to highlight the differences between what was actioned in the attack, and what was picked up by the IT team.

Before the test team arrives, it can also pay to undergo a security pulse check, including basic patch management and applying hardware and software updates. Of course, patch management and general updating should be regularly undertaken regardless, but ensuring everything is completely up-to-date as a preliminary for penetration testing will mean that security measures are tested in their entirety.

Prepare also to work collaboratively with the test team. Sharing knowledge beforehand can save them time, and you money. Conversely, you should take the time to understand the methods and tools they will use to analyse your network. The more you understand, the more valuable the test.