The offensive security industry has spent over a decade building methodologies from structured frameworks to step-by-step guides on how to assess a target. They’re generally useful until you realize what experts actually do in the field.
Since 2012, I’ve been fascinated with how experienced hackers approach engagements. I have worked with well over 200 hackers throughout the years, and when I ask about how they start engagements I keep hearing the same thing: “I used the application and I kind of knew where to look.” Note that they don’t say “I followed OWASP’s testing guide” or “I ran through my checklist.” They just…knew. When pressed deeper on how they knew, they struggle to explain it.
This gap between what experts do and what they can articulate is the foundation of formal research I’ve been conducting as part of my PhD studies, and I’ll be presenting my findings on June 24th in Virginia. Spoiler alert: The recognized experts I interviewed didn’t talk about methodologies; they talked about their past.
Bugcrowd’s Inside the Mind of a Hacker reports have consistently revealed something that the industry is still coming to terms with: Hackers don’t come from a single mold. They have wildly different backgrounds, from software engineering and IT, to self-taught tinkering and competitive CTF circuits. These differing areas of expertise don’t just make hackers great; they literally shape how they see targets. The data has been telling us this for years.
What the data hasn’t explained is why this diversity of experience matters at a cognitive level and how we capture it for better tooling, instruction, and improvements in our craft. Why does a former backend developer spot server-side issues faster? Why does a CTF veteran instinctively probe for edge cases? What are the decision environments and how do their thoughts drive actual work?
The study drew on a framework called Recognition-Primed Decision (RPD), originally studied in firefighters, military commanders, and emergency room doctors. The core finding from that body of work is that experts under pressure don’t weigh options analytically—they recognize patterns from prior experience, mentally simulate a course of action, and adjust if it doesn’t fit. It’s fast, intuitive, and almost invisible to the person doing it.
This maps to an older idea in skill acquisition research: that as people gain expertise, they stop following rules consciously. They develop something closer to intuition — a deep, compressed knowledge base built from hours of experience that they often can’t fully put into words because it just becomes the norm.
Both of these ideas seemed to describe my own experience and what I’d been watching hackers do for years. Therefore, I designed a study to find out.
I interviewed 15 recognized experts across three communities—bug bounty hunters, capture-the-flag competitors, and security consultants—using a method borrowed from cognitive science called the Critical Decision Method (CDM). Rather than asking what someone did during a notable hack or engagement, CDM focuses on what drew their attention, what the surrounding environment was like, what pressures they were under, and how prior experiences surfaced in the moment. It seeks deeper tacit knowledge.
Each interview followed four progressively deeper passes over a single, self-selected engagement. We built a timeline, layered in context (How far into the engagement were they? Were they under time pressure? Had they hit dead ends?), then probed specifically into attention cues, environmental influences, and past experiences that shaped real-time decisions. The final pass asked experts to imagine alternatives, and explore questions like where a less experienced person would have gotten stuck and why.
The most significant finding was that 12 of the 15 experts didn’t once reference methodology as a factor in their decision-making. What they described instead was a web of prior experience that had actively shaped their attention. They found patterns that resonated with prior experiences and capitalized on them.
A former software engineer had recognized a vulnerability class because they’d written that kind of flawed code before. A CTF player had approached a web application’s authentication flow the same way he’d dismantled a similar challenge in competition because the designer had followed a pattern they had seen. A consultant had focused their time in an area because they’d seen the same vendor stack have similar problems as previous clients.
In each case, their past was the engine of their attention and how they made sense of their tests. Their accumulated experience told them where to look, what to expect, and when something didn’t fit the pattern. That’s not methodology. That’s recognition.
The three experts who did mention methodology described it as scaffolding for organizing their workflow. More of a way to frame and report their actions, not as something they followed during the actual assessment. They talked fondly about how methodologies shaped their learning, but they never follow any methodology as a prescription for success. The methodology was for a deliverable, not for the result.
We’ve known for years that diversity of thought, experience, and intuition played a large part of what people did during assessments. But those discussions never explored the cognitive approaches specifically used to explain why those backgrounds affected testing decisions. If you’re building training programs, or even agentic harnesses for assessments, this finding has direct implications. Teaching beginners a methodology gives them a valuable starting framework but it won’t produce expertise. What yields expertise is varied, meaningful experiences that build the pattern library experts draw from unconsciously. Labs, CTFs, real-world exposure across different environments, different tech stacks, building software, different organizational contexts—that’s the raw material.
If you’re running a bug bounty program, this validates what platforms like Bugcrowd have been demonstrating in practice: The strength of the crowd comes from cognitive diversity. Different hackers bring different histories, which means they literally see different things when they look at the same target. A methodology can’t replicate what only experience can.
If you’re an aspiring hacker wondering why the checklist isn’t clicking, it’s not you. The checklist never actually dictated how experts operate. They just couldn’t explain what they were doing instead.
Now there’s research that can.
I’ll be presenting the full findings, methodology, and implications at the Naturalistic Decision Making Associations Conference on June 24th in Virginia. If this resonated, or if you think the research is wrong, I’d love to see you there. https://ndma.regfox.com/icndm2026