Our AI strategy for preemptive security
What happens when you put a senior platform security engineer and a director of red team operations in the same room and ask them the same questions? You get a refreshingly honest conversation with some uncomfortable truths sprinkled in.
Bugcrowd recently hosted its latest Ask Me Anything session featuring Alex Caughey, Senior Platform Security Engineer at Trustpilot, and Alistair G, Director of Red Team Operations at Bugcrowd. Together, they pulled back the curtain on what modern security programs actually look like in practice: what’s working, what’s overrated, and where elite attackers still find ways in.
You can view the whole session here, but we collected some of the highlights below.
Trustpilot already had EDR, SAST scanners, and regular pen tests when they launched their bug bounty program. So why bother?
For Alex, it came down to continuous coverage. “We’re shipping new code every single day,” he explained. “Pen tests are point-in-time assessments—the target is continually evolving.” Bug bounty offered something no scan or scheduled test could: a large, diverse pool of researchers assessing the platform 24/7.
Alistair echoed this from the offensive side. Even in hardened organizations, serious issues tend to hide in the seams, like in the assumptions made about trust between systems, teams, and third-party integrations. “Every vulnerability is potentially interesting,” he said. “It’s not ‘is there a bug?’ It’s ‘is there a path?'”
We asked what security control gives executives the most false confidence. Alistair’s answer here was deliberately uncomfortable. He believes it’s not any single control; it’s the belief that a control is complete.
“A dashboard that says 90% coverage… Fine, but what does that mean?” he asked. “Is an agent installed on 90% of assets? Is telemetry meaningful? Can someone actually respond on a Friday evening when one of your servers gets popped?” Passing a pen test, having a high EDR coverage number, achieving SOC 2—these are photographs, not posture. Code changes. People change. Infrastructure changes.
Alex agreed. He’d seen it himself. After launching the bug bounty program, findings appeared that no prior process had caught. “There clearly were some gaps in our understanding of where issues were.”
When asked about the most underrated attack surface, both Alex and Alistair landed on the same answer: identity.
For Alistair, it’s the trust boundary—who can speak to what, how identity crosses between systems, how CI/CD gets trusted by production, how API tokens bypass SSO. “In super mature organizations, the trade craft mainly evolves around compromising people who have particular access,” he said, pointing to threat groups whose tactics now center almost entirely on social engineering and credential theft.
Alex gave a concrete example: even with all internal tools behind SSO and MFA, developers could generate API keys inside those tools, keys that, if leaked, completely bypassed authentication. “It’s all about finding these small disclosures and leaks that have large permissions in our systems.”
A technically valid vulnerability and one that actually matters to the business aren’t always the same thing.
Alex shared a story: Trustpilot received what initially looked like a low-severity IDOR; an attacker replacing an ID to modify a metadata parameter. Innocuous at first glance, but that metadata parameter fed into the algorithm that surfaces results to users. What looked like a minor data tweak had potential implications for the integrity of Trustpilot’s core product and its reputation.
“The best researchers are the ones that really start to understand what’s important to the business,” Alex said. For Trustpilot, that means understanding that what’s being protected isn’t just data… it’s trust itself.
AI is making researchers faster at reading code, summarizing unfamiliar systems, writing tooling, and structuring findings. Both panelists agreed on that. Alistair shared thoughts about where it goes wrong: when it’s not validated, not grounded in evidence, and not given sufficient context.
From the triage side, Alex noted that Bugcrowd’s initial triage layer means Trustpilot sees mostly validated, well-written reports. But he’s noticed a recent shift: researchers are using AI to write more clearly, align findings to the threat model more precisely, and communicate impact more effectively. “The way these vulnerabilities are explained is probably getting better.”
The final question landed on something that matters for any program operator: how do you keep your best researchers engaged?
Alistair’s answer focused on transparency. Don’t close off vulnerabilities mid-engagement without communication. Don’t play games. “The primary thing we care about is making each organization as mature as possible.”
Alex was candid about early mistakes. When Trustpilot first launched, the volume of submissions overwhelmed their response times and slow feedback frustrates researchers. The fix was practical: integrations between Bugcrowd and their work management and messaging tools so that new submissions, comments, and updates triggered immediate internal alerts.
Check out the full conversation which includes a deeper dive into AI-assisted exploitation, proof-of-concept standards in the age of AI-generated reports, and what red teamers are actually looking for.