The concept of red teaming has been around since the Cold War, but companies still find it prohibitively difficult to hire a flexible, yet thorough, red team. Red team consultancies are commonly used, but they’re not always equipped to deal with the specifics of a specific organization’s attack surface. To make red teaming more accessible, Bugcrowd launched Red Team as a Service (RTaaS) to harmonize the thoroughness of red teams and the flexibility of crowdsourced testing. With Bugcrowd’s RTaaS, companies can work with red teams on their own terms by choosing between Assured, Blended, and Continuous red teaming options. 

This industry-first model was built by red teamers to address some of the market’s most complicated problems. In this blog post, we’ll give you a behind-the-scenes look at how Bugcrowd builds the world’s most talented, cutting-edge red teams—groups of vetted experts that will help you stay ahead of real-world attacks.

The qualities of a leading red team

To make a great red team, we first needed to determine what “great” looks like. From discussions with veteran red team operators and with companies that raved about their red team experiences, we consolidated a thorough, pages-long list of characteristics. Some of these qualities (e.g., professionalism and ethical behavior) are table stakes for a red team, while others will take an engagement to the next level. Among the most important qualities we look for in a red teamer include the following: 

• Collaborative—We’re sourcing red teamers who talk among each other frequently during an operation and truly partner together. They discuss alternative approaches to attacks, immediately notify each other of successful and failing exploits, and frequently plan and replan their next move. The best red teamers work together on compromising hosts, targeted phishing campaigns, and ultimately, achieving the objectives.
• Communicative—A great red team proactively corresponds with clients about its plans and attacks, decisions it makes during an operation, and the resources it targets, compromises, and exfiltrates. This applies to both spoken and written communication, the latter being especially important in creating post-op debriefs that are actionable and prioritized.
• Broad-spectrum—The red teamers we work with can find vulnerabilities across a company’s whole attack surface, technology, people, and processes. With a broad spectrum of skills, red teamers can chain attacks from Active Directory and cloud to mobile and AI to find vulnerabilities others can’t.

With these qualities as the foundation, we created our red team selection and formation process. We’ll dive into that next.

Our red team formation process

To create collaborative, communicative, and broad-spectrum red teams, we built a process where we first select only the most competent and diligent red team operators. Then, we form red teams by looking at skill and experience fit among team members. We don’t expect one individual red team operator to have 100% of the skills needed for a specific engagement. Instead, we compile teams that can cover all of the skills needed and more. Finally, we get the red teams operation-ready through live simulations.

This process ensures that customers who invest in a red team engagement truly get the best of the best. Customers can rest easy knowing that a red team curated for their exact circumstances is busy uncovering evolving attack chains in their environments, just like real cybercriminals would. 

Initial skills and experience screening

Once a red team operator applies to work with us, the first thing we do is understand their background. We take into account not only their technical skills and work experience but also contextual factors like regional expertise and personal drive. We use this information to categorize applicants into three roles and three tiers of red teaming expertise, which helps us form an array of broad-spectrum red teams to cover our customer’s needs. 

This section goes into detail regarding our exact process when building red teams. In summary, when Bugcrowd is building a red team, we want to find out the level of seniority, the role, and the skill sets of a potential team member. These are determined by assessing many factors, including experience, certifications, time zone and working hours, skills and technology familiarity, geographic and industry operation history, their ability to bypass different layers of security controls, and the illusive X factor. 

Specialist, operator, and manager roles

By assigning separate roles to red teamers, we can better understand what they bring to the table. While all operators can perform hands-on-keyboard activity, we use more explicit and nuanced definitions:

• Red team specialist—Specialists have deep domain expertise (e.g., advanced knowledge of cloud exploitation, social engineering, or malware development) to help teams tackle specific attack surfaces but may be brought into only one phase of a red team operation.
• Red team operator—Operators execute attacks against various technologies, people, and processes. They are the generalists on a red team and have experience running full-spectrum assessments.
• Red team manager—Managers coordinate and control operations during a live red team assessment. They oversee tactics, techniques, and procedures (TTPs) deployment, operator tasking, and critical decision-making to ensure an assessment succeeds without violating policies or laws. The manager is usually a highly experienced operator (commonly a Certified Simulated Attack Manager, or CCSAM).

We may occasionally have junior operators or specialists shadowing or deploy dedicated developers when developing capabilities may be necessary (i.e., in hardened environments).

Red teaming experience levels

Below, we list the tiers and key indicators we look for. These tiers, helpfully named after levels of spice, help us understand a red teamer’s skill level. 

The idea behind the tiers is for them to help us answer the following questions: 

1. What tools and areas of the attack chain is the operator familiar with?
2. Does the operator know what attack and/or tool to use and when to use it?
3. Does the operator understand the OPSEC and risk considerations of running the attack or tool?

Answering these questions allows us to understand the level of exposure an applicant has to red teaming. It’s important to note that these tiers aren’t necessarily gospel. For example, a red teamer might only have one of the skills in the list, and that is okay. These tiers provide us indicators of applicants’ understanding.

Tier 1: Starting out red

In the first tier, red team operators have a solid understanding of red teaming and sometimes demonstrable programming skills. They can execute most attacks in a red team assessment. A red teamer in this tier can successfully perform an Active Directory exploitation, leveraging tools like BloodHound and Mimikatz to map user permissions, escalate privileges, and move laterally within an enterprise network to identify and exploit security gaps. However, they may not be fully up to date with modern tradecraft, be able to create novel attack vectors, or plan red team campaigns.

Indicators of this tier

• Hands-on labs and CTF challenges (Hack the Box Pro Labs Dante and RastaLabs)
• Red team certifications (CRTO, CRTP, CRTE, or TCM PJPT)
• Programming skills (C/C++, SQL, Java, Python, PHP, x86, or HTML/CSS)
• Experience with internal network pen testing, OSINT, phishing, or red team operations.

Tier 2: Spicy

In the second tier, red team operators not only have all the foundational knowledge and skills of Tier 1 operators, but they also have deep knowledge of red teaming tactics and specific attack surfaces. They’ve participated in many red teaming simulations and usually have advanced certifications. They can formulate and chain novel attacks and know how to run them across various environments. A red team operator in this tier knows how to abuse identity and access management (IAM) roles and conditional access bypasses, as well as how to use and write tools to bypass endpoint detection and response (EDR) defenses to gain domain admin privileges. However, they may not be able to manage a red team, plan an engagement, or do modern exploit development against hardened systems.

Indicators of this tier

• Hard hands-on labs (Hack the Box Cybernetics or Breaching AWS Labs)
• Advanced red-team certifications (OSEP, CRTO II, OSED, or HTB CWEE)
• Experience on many red team operations.

Tier 3: Flamin’ Hot

The third tier indicates a high level of autonomy and expertise in red teaming, as well as a good understanding of techniques and how to evade modern defenses while minimizing impact to clients. 

In this tier, red team operators possess the same deep knowledge as Tier 2 operators. They can architect full sequences of attacks while building custom attack vectors and writing exploits for each vulnerability. Additionally, they have the skills and experience to lead a whole red team to a successful operation. A red team operator in this tier can architect and oversee a multistage, cross-domain adversary simulation by designing bespoke exploits, tools, and attacks; coordinating a team to execute complex kill chains; adapting tactics on the fly to address evolving defenses; and guiding analysis and reporting.

Indicators of this tier

• Pro-level labs (Hack The Box Pro Lab APT Labs)
• Elite certifications (OSEE, CCSAM, and CCSAS)
• Advanced training (Maldev Academy)
• Experience with red team operations in complex environments and with regulatory oversight.

Context matters

We also take note of some extra context around an applicant’s skills and experience to inform later team formation. This context can take many forms—from applicants being OPSEC-aware and the time zone they are in to the geographical experience they have. 

It might be surprising that we consider applicants’ geographical experience, considering that red teams operate remotely. However, the particular geographical location of an engagement requires red teamers with knowledge of specific regional laws, cultural sensitivity, and physical safety indicators. For example, particular reconnaissance tools might be illegal to use in Germany, so for an engagement in Germany, we’d connect the customer with a red team with relevant experience. Another example is the importance of considering cultural sensitivity factors when sharing a final report and debrief. If the client is a CISO in Japan, a good red teamer will understand that the debrief should take a different tone than one written for a UK audience because Japan has different norms for pointing out potential issues with a security strategy. When constructing red teams working with Japanese companies, we would prioritize applicants with prior experience.

Operator location and other context cues allow us to further optimize our client services. From adhering to regional working hours to keeping up to date with the local news, customized services provide clients with the best ROI.

A red teamer’s X factor

The consistent ability to go above and beyond is the most promising quality we can find in a red team operator. An excellent red teamer is curious and has an insatiable hustle mentality, which means they will often do whatever it takes to help a customer, including learning an entirely new attack surface. They will find more vulnerabilities and help more companies improve their security postures. During the screening process, this quality shows up in many ways: engaging in red teaming as a side hustle, working early or late hours to help a client in a different time zone, or continually achieving the next level of certifications.

There isn’t an exact science to measuring this X factor, but another indicator is someone who speaks at industry conferences, publishes blogs, authors tools and techniques, competes in CTFs, and pushes the limits of hacking. Why do these activities matter? Simply put, they mark the difference between someone who will give up after something doesn’t work the first few tries and someone who is persistent. 

In-depth solo simulation

At this point in our process, a red teamer might look like a good fit on paper. We now test them to verify their knowledge and skills. Each applicant at this stage will run through an online simulation, where we’ll present each applicant with red team tradecraft—the TTPs—and, for each one, ask the following:

• When would you use this TTP and what is it?
• How would it need to be modified for different environments?
• What are the expected results and potential obstacles to using each TTP?
• What are the OPSEC and risk considerations around using that TTP? 

Finally, each applicant will have to use multiple TTPs to simulate an attack without getting caught.

For all applicants who pass this step, we run a background check to verify their financial, criminal, and employment histories. This ensures that customers are in trusted hands.

Team formation and simulation

Now that we have a pool of exceptional, verified red team hackers, we form the teams. Each team will have managers and operators to cover the entire kill chain. Specialists can rotate between teams as needed to provide their advanced skills. Typically, teams consist of 2 to 5 individuals. To ensure that team members can collaborate effectively and cover a wide range of attacks, we group folks with complementary skill sets who can work across similar time zones.

Beyond testing each member’s skills, we also want to ensure that all team members will work well with each other and with clients. So, we put teams through a simulated operation and grade them on a thorough rubric. Some criteria include formal written communication, research, problem-solving, teamwork, commercial awareness, adaptability, and skill compatibility. Teams are only permitted to proceed together if they earn high scores across all of these areas. A red team is only considered operation-ready once it has passed this final simulation.

The entire team construction and vetting process is incredibly thorough—for good reason! It prepares red teams to use the Bugcrowd Platform to communicate findings, leverage the attack approval process, communicate covertly with clients, and digest cyber threat intelligence. Bugcrowd also provides a “bring your own tenancy” infrastructure management model. In this step, red teamers will get experience using both standardized infrastructure as code and customized setups. This provides operators with everything they need to perform world-class red teaming. 

Matching customers with the right red team

The last step is matching customer engagements with the right red team based on their needs. This requires understanding the overall skill sets and specialties of the team and making sure that it fits with the requirements of the engagement. Here, we rely on our CrowdMatch algorithm, which analyzes your attack surface along with the skills and previous engagements of our red teams to align assets and needs. CrowdMatch leverages preliminary reconnaissance and cyber threat intelligence, which enables us to understand the likely attack vectors facing an organization. That way, if we see on a LinkedIn Jobs advertisement that an organization uses Microsoft 365 and Windows endpoints with an internal active directory, we can provide it with experts in those areas, as opposed to sending in the MacOS folks.

We also filter red teams by time zone to make communication between customers and the team easy, as well as making sure they blend into business as usual. Finally, our internal specialists will also handpick teams to ensure the best fit. As a result, customers get a red team that can handle their engagements, no matter the models they choose.

To recap, Bugcrowd’s RTaaS offering is the first of its kind, helping customers find the right red team for their needs to stay ahead of real-world attacks. To pull this off, we’ve built a rigorous process to find and select the best red team operators. Customers can easily scale their engagements up or down by involving Bugcrowd’s global pool of vetted red teamers and multitiered options to ensure fresh tactics and up-to-date skills. As a result, customers can be confident in the thoroughness of their red team assessments while knowing that they got one as efficiently as possible. To learn more about how organizations are elevating their security maturity with RTaaS, download The Ultimate Guide to Red Teaming.