This year, one of our favorite customers will be speaking at one of our favorite conferences where they will discuss why they implemented a bug bounty program, and how the results and learnings have influenced their internal security culture and testing processes.

If you will be at LASCON in Austin, Texas this year, be sure to catch Charles Valentine, VP of Technology Services at Indeed, talk on Friday at 1 pm. If you won’t be at LASCON, we have you covered. This post offers a preview of what went into the launch Indeed’s bug bounty program and provides a snapshot of the results and findings to date.

Why a bug bounty for Indeed?

In mid-2014, Indeed recognized that they needed to create a consolidated channel for vulnerability reporting for researchers and improve internal and external security testing practices. To achieve this, they tapped Bugcrowd to help them leverage the power of the crowd and launched their bug bounty program May 22, 2014.

“We always consider the security of our systems as we develop the services that millions of people use every day. But someone will outsmart us. Hackers are always trying out new ways of bypassing security and gaining access to systems and information. Our challenge: to bring these security experts over to our side and benefit from their findings.”

Read more about their experience launching a bug bounty program in their recent blog post.

Snapshot of results from their program:

Since May of 2014, they have gotten more testing coverage than ever, uncovering more unknown vulnerabilities than they could have possibly done with their existing testing resources.

In the two and a half years that Indeed has run their bug bounty program, they have…

• Received over 3,000 submissions from researchers in over 60 countries
• Fixed nearly 500 valid bugs ranging in criticality
• An average priority of 3.5, which is better than that of all programs averages
• Paid out over $100,000 with an average payout of $264

Indeed has demonstrated tremendous commitment and success. Not only have they received consistent and high-level activity for a sustained amount of time, but they have also garnered trust and loyalty for several of the top researchers in the community, as proven by their nomination in our recent Buggy Awards for ‘Best Program – Researchers’ Choice.’

They have demonstrated transparent communication with submitting researchers, have increased payouts over time, and have committed to responding quickly to researchers, resulting in the volume and quality of submissions they’ve received.

What is more, because of their longstanding program, they have identified trends that have enabled them to evolved and improved their program over time.

Learnings from two years of bug bounty findings:

We’ve seen that over time bug bounty programs naturally fluctuate up and down depending on changing program variables. Indeed’s program is no different.

For example, if you split up their program in two, comparing the first half of their program (May 2014 to August 2015) to the second half (August 2015 to today) some basic trends emerge.

• Average priority improved: In the first half of their program, the average priority of valid submissions was 3.6, and improved to 3.3 in the second half. Over time, as more common or less critical vulnerabilities are found, Indeed’s security team has not only fixed those vulnerabilities but has learned how to avoid making those same mistakes. Over time, more high-value than lower value submissions have been made.
• Average payout increased substantially: From $160 in the first half of their program to $375 to the second half, this increase is a true sign of increased value. By increasing rewards, Indeed was able to incentivize more top testing talent, and receive more high-value submissions.
• The signal-to-noise ratio decreased: Although submission volume decreased, the percentage of valid vulnerabilities went up significantly. In the first half of their program, the program’s signal-to-noise was 13%, compared to 18% in the latter half. This is the goal for bounty programs—to have as much value as possible—and is best achieved through deliberate scoping and articulation of what is and isn’t in scope, as well as continual communication.

In two years, Indeed has done a great job keeping researchers engaged, while making more out of their continuous investment in the researcher community. They have increased their payouts, have made adjustments internally to process submissions more effectively, and have constantly committed to making their bug bounty program even better.

“…we’re working on balancing the time we spend finding new bugs and fixing known bugs. Building and managing a popular bounty program leads to lots of good submissions, but that all falls to pieces if we don’t also spend the time fixing the bugs. At Indeed, the benefits of investing time improving our bug bounty program can’t be overstated.”

Read more about their bug bounty learnings in their recent blog post.

Learn more about their success, catch Charles’s talk this Friday at LASCON, and stay tuned for the recording of Charles Valentine’s insights and advice for those looking to run a bug bounty program.