Bugcrowd launched Next Gen Pen Test, the first product in our Pen Test portfolio, in November of 2018. Since then, “The Difference Between Bug Bounty and Next Gen Pen Test” has remained one of our most popular blogs. Maybe not surprisingly, as prior to that, the terms crowdsourced security, bug bounty, and pen test were often used interchangeably. That blog has helped countless security leaders disambiguate these terms, while serving as a strong first step in choosing the product that best fits their business needs.
But 18 months on, we’ve learned that sometimes not choosing, may be the right choice. That is to say while we’ve helped address a wide range of use cases, including replacing traditional pen testing with Bug Bounty, or swapping Bug Bounty for Next Gen Pen Test, it turns out companies that run both products (where appropriate) have seen some of the most significant gains in submission volumes, long-term researcher engagement, and total cost savings.
In this blog we’ll revisit the differences between two of Bugcrowd’s most popular product portfolios, how to choose one over the over, and the top 4 use cases where combining the two might make the most sense for your business.
Why Choose Bugcrowd Bug Bounty over Traditional Pen Testing?
Bug bounties (as a generalized term) provide quick access to varied security expertise, in a “pay-for-success” rather than “pay-for-time” model. Pen testing (as a generalized term), leverages humans to manually test the security of an asset. Unsurprisingly, this is why some organizations, especially those without strict compliance requirements, consider bug bounties to be a form of pen testing– at least as far as their budget is concerned. For these organizations, bug bounties are more than a sufficient replacement for their testing requirements.
Bugcrowd Bug Bounty, in particular, provide faster time to value through multi-factor dynamic skill matching and recruiting, real-time platform reporting, fully managed triage, and developer tool integrations like JIRA and ServiceNow, along with vulnerability remediation advice to fix fast. These features help security teams reduce time in triage and consultation, while increasing coverage, and number and severity of findings.
Bug Bounty can also be deployed on-demand– in a time-boxed testing window– or on a continuous basis. And with the average time to first critical vulnerability clocking in at just 2.7 days for the average On-Demand Bug Bounty program, security teams purely looking for swift, impactful testing really needn’t look any farther.
Why Choose Bugcrowd Pen Testing over Traditional Pen Testing, or Bug Bounty?
Many of our customers have made a permanent shift– trading traditional pen testing for the flexibility and impact of fully managed Bug Bounty programs. But this substitution isn’t necessarily appropriate for those that define pen testing a little differently. Many still have strict compliance, budgetary, and procurement requirements, or need a level of testing “predictability” that skill-based competitive bug bounty programs can’t quite meet.
To bridge this gap, Bugcrowd launched the first of our Pen Test portfolio solutions, Next Gen Pen Test. NGPT provides methodology-driven security testing, with a “reward pool” dynamic that encourages greater speed to launch, and more impactful findings, giving organizations the compliance artifacts they need, with the results they want. Additionally, depending on tier of service selected, NGPT includes retesting, coverage analysis, premium SLAs, executive reporting, and can be deployed on a time-boxed, or continuous basis. Bugcrowd’s Next Gen Pen Test program and final compliance report have also been assessed by an independent QSAC to meet PCI, ISO-27001, and NIST security requirements.
For organizations that prefer a more predictable pricing structure due to procurement or budgetary cycles, Bugcrowd recently introduced Classic Pen Test (CPT). CPT provides the same platform-level features like rapid and dynamic skill matching, streaming vulnerabilities, and managed triage, without the variable “reward pool” of NGPT.
All Bugcrowd Pen Test programs provide a standardized view of security posture in a reporting format your auditors, partners, customers, and investors have grown to understand, making either option an obvious choice for organizations looking for more structured testing on priority targets.
The Power of Bug Bounty and Pen Test Together
Organizations are complex, and not all applications are treated equally, especially when distributed across several teams. While there’s something to be said for a standardized approach, there’s really no such thing as a ‘standard’ asset– which often results in (intentionally, or unintentionally) fractured security testing programs. In the case of bug bounties and pen testing, there’s a few good reason why you might need both:
Scenario 1: New Stakeholders
As businesses evolve and expand, new stakeholders in the way of partners, investors, board members, and new customer personas begin to enter the sphere of influence. With increasing variety in the needs and expectations of each, businesses need a way to more effectively communicate critical insights and best practices in a format that can be easily consumed by all.
Running continuous or multiple Bug Bounty programs across critical assets may be well understood and appreciated by more security-minded audiences, though there is much to be said for a more standardized format which even non-security audiences can readily understand. Bugcrowd customers with this use case often compliment their Bug Bounty program with a strategic insertion of Pen Test on mission critical assets that are more frequently referenced, presented, pitched, or assessed to provide standardized, executive-level reporting on-demand.
Scenario 2: Mergers and Acquisitions
M&As are incredibly complex and often lengthy engagements– but not for the security teams brought in just weeks before the deal is signed. With little time to assess an acquisition target, speed to deploy and ability to provide clear recommendations are paramount when it comes to selecting a testing provider. Unless egregious, volume and variety of findings have become a secondary concern, as many can be resolved post-sale, and richness of results hasn’t historically been associated with speed of delivery.
This often left security leaders with two choices:
- Face lengthy pen test scheduling delays (unlikely to be tenable given short timelines)
- Prepare to pay premium prices for expedited services (unlikely to result in exceptional results even with price paid)
- Launch a time-boxed bug bounty (not a bad idea, but the goal of a structured, defensible, and methodical assessment can be at odds with the competitive nature of a bug bounty, especially if only one particular aspect captures the most attention)
A better option:
With Bugcrowd, organizations can quickly deploy methodology-driven testing in as little as 72 hours. With options to incentivize testers, organizations no long have to choose between speed, reporting, and quality of results. Now through a single platform, customers can deploy Pen Test and Bug Bounty for pre-sale due diligence, and in-depth post-sale continuous coverage
Customers with this use case have particularly benefited from speed to launch, as well as the impact of information sharing between pre and post-sale analysis. Pen Test outcomes can provide critical context for researchers in the Bug Bounty program, further reducing time to value and enriching outcomes.
Scenario 3: Compliance for a subset of assets
While some regulatory standards apply to all of an organization’s assets, others impact only a subset. The Payment Card Industry Data Security Standard, or PCI-DSS is a good example. PCI-DSS requires organizations to apply methodology-based pen testing to applications that process or store credit card information (specifically the cardholder data environment, or CDE), but not those that don’t, like a chat app, or customer log-in page. If the organization is interested in an integrated testing strategy that covers all assets, they previously had a few options:
- Engage a single pen test provider. This provides the audit-ready reporting needed for a payment app, though, finding the skills needed to effectively test other assets of varying complexity could result in lengthy scheduling delays. Additionally, “single” vendor isn’t quite accurate, as best practice dictates organizations cycle pen testing providers to gain fresh perspectives.
- Put all assets in bug bounty programs. While bug bounties launch quickly and provide continuous access to new testers, organizations seeking compliance could run into trouble with auditors less familiar with this testing style.
- Leverage a traditional pen test shop for the payment app, and run a bug bounty program for the others. A good compromise in theory, though, managing multiple vendors with unique testing and reporting workflows can be burdensome on security and development teams alike. This also doesn’t quite solve for the pains of traditional pen testing.
A better option:
Fortunately, there’s one more option, that doesn’t compromise on speed, quality of results, or ability to meet compliance objectives. Customers with this use case deploy Bugcrowd Pen Test and Bug Bounty solutions together, either on-demand or on a continuous basis, on separate targets depending on reporting priority. All programs are then managed through a single vulnerability management and reporting console. This arrangement helps reduce procurement complexity, eliminate operational inconsistencies, and improve outcomes across the board.
Scenario 4: Customer Acquisition
Your customers take security seriously. If you can’t demonstrate the same, you may lose business to a competitor who can. To assess vendor security posture, almost all organizations now perform cross-functional IT and Security vendor assessments prior to purchase. This process typically requires proof of a recent pen test (among other security controls).
Of course, time is a critical factor for the seller in such situations. Due to lengthy scheduling delays for more reputable testing providers, many sellers now hastily seek ‘quick & cheap’ alternatives to tick the box and progress the deal. Such tactics are increasingly diluting the ability of a single pen test report to serve as an accurate representation of an organization’s broader security posture.
A better option:
Hopeful sellers that can furnish both a standardized assessment as well as evidence of a continuous, layered approach to security testing may now be better placed to win the favor of increasingly security-minded customers. When faced with buyer requirements like these, Bugcrowd customers like SoftDocs, or ActiveCampaign point to parallel Bug Bounty and Next Gen Pen Test programs that illustrate their commitment to regular testing and timely remediation. As Chaim Mazal, Head of Global Information Security at ActiveCampaign notes in a recent case study, “You can’t be #1 if you’re not the #1 most trusted.”
Growing to Meet Every Security Testing Use Case
The variety of security use cases faced by Bugcrowd customers is on the rise. For organizations that plan to increase product offerings or expand into new markets, the addition of increasingly diverse stakeholders drastically impacts how and why security testing is performed. There is no more ‘one-size fits all’ approach.
The Bugcrowd platform was designed to enable organizations to combine and configure product workflows and feature sets to meet increasingly diversified security needs today, and well into the future. For more information on any of the Bugcrowd product portfolios, including Bug Bounty, Pen Test, Vulnerability Disclosure, and Attack Surface Management, visit our website, or get started today!
