Can bug bounty programs replace penetration tests?

This question has come up a lot in the past several months and today we released a guide that begins to answer it.

In this resource, we unpack key differences (and key similarities) between the two, including the testers, the coverage, the incentive model – and ultimately, the results.

The term penetration testing has come to mean a lot of things… From a simple scan with cursory human validation, all the way to highly tailored red team engagements. As a former pen tester and pen test firm proprietor, I’ve seen and befriended many firms that I’d be proud to have as my security backstop, but I’ve also seen the low-quality bar that the market has learned to accept.

I do think that we should differentiate these by definition, but I also think that the penetration testing model is fundamentally flawed, and here’s why…

  • How many actors put how many hours into probing your systems for vulnerabilities? (Hint: The answer is right there in your logs).
  • Are those actors incentivized by their results, or are they more incentivized to “get the gold” in your organization? (Hint: Sometimes… But the majority of actors are results-driven)
  • Is paying one or two people, no matter how competent they are, for forty or eighty hours of their time going to put your company in the position to be able to compete with a crowd of bad actors to find the flaws first?
  • The current pen testing model can’t be continuous, and thus, can’t keep up with agile development.

In short, it isn’t working… and yes, it can be replaced by crowdsourced security assessments in many instances. Bugcrowd was the first to offer point-in-time crowdsourced bug bounties, and we have continued to flip the pen test model on its head through our fully managed solutions, bringing researchers and companies closer together to achieve continuous security coverage and improve ROI.

In the upcoming weeks, we’ll address many misconceptions about the penetration testing space, and get perspectives on the subject from industry leaders and penetration testers past and present. Some common questions we’ll answer include but are not limited to…

  • Can you trust bug hunters?
  • How do you ensure full testing coverage across technologies?
  • How can you control testing and total spend?
  • Are the results on par with penetration testing?
  • Is the level of liability higher with bug bounties?

We welcome your feedback in this discussion and hope to include as many individuals in this conversation as possible. Please feel free to reach out to hello@bugcrowd.com or @Bugcrowd to join the discussion.