Post by P3T3r_R4bb1t
As a hacker, I am motivated to help program owners reach their goals. I’ve seen a lot of great programs in my eight years of bug bounty hunting, but I’ve also witnessed recurring mistakes made by program owners. These mistakes not only created frustration, they completely steered me away from certain programs and toward other playing fields. Using my perspective as a hacker, I created this short list to educate current and future program owners on the most common mistakes that discourage hacker engagement.
I used to work as a practice lead for a consulting firm a couple of years ago, and one great friend of mine used to say that “scope is only a suggestion.” I still believe he is right. To be successful as an ethical hacker on a professional engagement, you interestingly need to learn how to slightly bend the rules in your favor, without putting yourself into a loophole. However, in the bug bounty space, that’s less true. Safe harbors are designed to protect hackers while attacking the defined scope, and if you go out of scope, you can face some severe legal consequences.
That’s fair, but what if the scope is not clear? Very often, I have seen programs stating their main site URL as the scope as a top level domain (e.g., domain.com) but without the wildcard, without anything after the domain name. Does that mean I can hunt on all subdomains? What if the site is relying on an API hosted on api.domain.cloud instead? Does it count? You can immediately see the scope hasn’t been well defined, and that will drive hackers to hold back on some potentially valid submissions to avoid out of scope deductions. Without going into all specific details, program owners must be able to clearly state what’s in and what’s out in order to have a successful hacker experience.
Picture this: To drive engagement, a program owner decides to offer very appealing bounty ranges. This strategy aims to entice the best hackers to spend more time and find the most impactful issues. It works! A great hacker chooses the program, spends several hours figuring out a complex vulnerability, submits it to the program, and the triage team confirms it’s a solid P1. Unfortunately, after internal review the program owner and team decide to accept the submission, but instead of rewarding at the top of the range, they reward at the lowest band because, you know, budgets are tight.
Here’s why this bounty will create frustration and drive hackers away from programs instead of creating satisfaction and loyalty toward a program: by setting a range, you create expectations. Instead, I would advise not to set a range and to provide a single, fixed number. This will set the baseline and create a clear expectation. If the issue is severe enough, you can add a bonus to that submission, which will create additional satisfaction and will motivate the hackers through incentives.
The endless CVSS debate is a recurring classic. Fortunately, the Bugcrowd Platform does not embrace that scoring framework by default. Bugcrowd uses its own VRT framework to provide more flexible scoring of vulnerabilities. However, I have seen programs in the past that feel it is necessary to align with CVSS.
Here’s why I don’t recommend that: the metrics of CVSS are too easy to manipulate to anyone’s advantage. That goes both ways. Hackers can tweak findings to make a vulnerability look worse than it is, and then program owners can also interpret the metrics to do exactly the inverse, which creates an inevitable titan clash. When confrontation happens, hackers will lose their motivation and go hunt elsewhere and program owners will lose engagement. In other words, if you want things to move smoothly on your programs, do not leverage CVSS. Use the VRT with documented deviations, if any!
On the Bugcrowd Platform, clients have the ability to exchange private messages with the triage team. This might sound like a great way to communicate, but put yourself in the hacker’s shoes for a minute. On our side, we see all the back and forth between the program and the triage team and have literally no idea what is being discussed. It simply creates uncertainty: what is going on? Is my bug valid or not? What are they saying? Why can’t I see these messages? These are some of the questions that will be raised in our minds.
After years of hunting, I personally label this situation as “the ping-pong game.” I have a very firm opinion that this should be avoided or minimized as much as possible. I believe that transparency should be prioritized and the hacker should be kept in the loop during communication. We are curious people by nature, and we can easily provide assistance or clarity if something in the report is confusing or needs to be adjusted. This is straight from the horse’s mouth, as they say. Having open communication with the hacker will drastically decrease all potential misunderstandings and will help us track the movement of our submissions, minimizing frustration and confusion.
The most irritating moment for me as a hacker, and I am sure my fellow hunters would back this up, is when a program lowers a bug severity, but doesn’t provide any explanation or rationale for that decision. This feels like receiving a brick to the face. Of course, hackers will never complain about a severity upgrade! However, I believe that even though clients have and will always have the final say in a submission, the bare minimum expectation is to provide at least a summary of why this downgrade happened. Maybe it’s context related, maybe it’s because of compensating controls that the hacker is not aware of, or maybe it’s because of the business’s risk profile.
Most hackers are cybersecurity professionals and are very understanding of strategic business decisions. Hackers have a lot of professional and life experience: They know what a risk is and have seen hundreds of flaws in their careers. They need to know why their assessment is incorrect, regardless of what is behind the downgrade. Providing a rationale or explanation will not only provide a learning lesson for both parties, but it will remove a lot of frustration and will hopefully maintain an open dialogue in future submissions.
I hope this article helps program owners recognize the main elements that lead to a reduction in hacker engagement and adjust their approach to maximize the value of their investments. At the end of the day, hackers are program owners’ most valuable allies, and program owners should treat them as such. Learn more about best practices for working with hackers by downloading Bugcrowd’s Guide to Working with Hackers.