We’re regularly asked how Bugcrowd determines if a bug bounty submission is rewardable. Today, as we approach 10,000 submissions, and as part of Bugcrowd’s commitment to transparency, we’re shedding some light on our submission evaluation process.

Its important to note up front that Bugcrowd programs differentiate between technical validity and rewardability, in order to maintain fairness to researchers and organizations alike. This means our customers only pay a reward for issues with impact to them and researchers get solid technical feedback about their submissions.

We’ve found its not uncommon to have a submission that is technically an issue, but lacks the impact necessary to reward it. In other words, it’s an ‘acceptable risk’ to the organization. Conversely, we’ve handled situations where features were ‘as-designed’, but turned out to be major security flaws that were fixed. This process has helped our customers determine what to do in those cases. If you’re thinking about starting a program, we’d encourage you follow this process and get in touch if we can help.

The Golden Rule(s): If you touch code or configuration because of the submission, reward the researcher.

Here’s the detailed process:

1. Is this a security issue? if not, mark as invalid and explain.
2. Does the bug bounty submission adhere to the terms and conditions? If not, mark as invalid and explain.
3. Was the submission communicated as an exception in the program brief? If so, mark as invalid and explain.
4. Has the submission, or its root cause been reported previously? Duplicates can and should be traced back to the code or configuration change that would resolve the issue. If that change has already been made, is in the queue to be made, or has been accepted as a risk, mark the issue as a duplicate and explain.
5. Is the submission technically reproducible? If not, mark as invalid and ask for clarification if you think it has technical merit. If so, it should be communicated as valid and you can move to determining the reward.
6. Will it cause you to make a code or configuration change now, or in the future? If so, it’s rewardable within the terms specified in your bounty brief. Issues with more impact should be rewarded at a higher level. Additionally, if it’s noted in the focus areas for the bounty, it’s worth more. If it does NOT cause you to make a code or configuration change, then provide reasoning to the submitter, and to the extent possible, push it to the brief as an exclusion for future testers.

It’s very important to consider the work of the researcher in step 5… If you think of other areas in the target that are impacted, or your security posture has been improved by discussion from the submission, we encourage you to reward it. If you’re on the fence, push the submission back to the researcher and ask for more information or how they view the impact. Crowdcontrol makes this easy. Remember, the researcher put effort into finding it for you, and it’s in your benefit to work closely with them and encourage them to go further.

The Golden Rule, transparency about your choices, and proactive communication should guide your judgement at all times.

This process has been informed by our own experience, and that of others in the community. As always, we’re open to comment and input. Let us know what you think!

If you’re thinking of starting a vulnerability disclosure program, or want to lighten the load of your existing program, Crowdcontrol is for you. Please get in touch if we can help!