When coming across a *.target.com scope, it’s always a good idea to seek the road less travelled. Exotic and forgotten applications running on strangely named subdomains will quickly lead to uncovering critical vulnerabilities and often high payouts. Discovering such subdomains is a critical skill for today’s bug hunter and choosing the right techniques and tools is paramount.
There are many techniques for subdomain discovery, from utilizing public resources such as Google or VirusTotal, to bruteforcing them, and sometimes also scanning an IP block and doing reverse lookups. Any of these methods will deliver OK results, while a combination of them ensures a more comprehensive output.
Today Bugcrowd will discuss some of our favorite discovery techniques. Come join the discussion on the Bugcrowd Forum!
Public Resources for Subdomain Discovery:
There are a number of public resources that cache subdomain information. Some of these sites are created for that specific purpose, while others such as search engines index them because that’s what they’re designed to do. Here’s an example:
Siphoning subdomains from Google is an easy job with Google Dorks. The site directive will filter results only to your target:
After we have the initial domain in there we can use the -inurl directive. Each subdomain we find can then be filtered out with more -inurl directives to make place for others:
After noting and iterating through all possibilities (until we get no more results from our searches) we get with a great starting point of subdomains. To do this manually is a pain but, there are tools that help automate this and also query multiple search engines at the time.
Public Resources That Can Contain Subdomains:
- Search engines (Google, Bing, Yahoo, Baidu)
- https://virustotal.com/ – Search for “domain:target.com” and virustotal will provide extensive information in addition to Observed subdomains, which is a list of all subdomains it knows about.
- https://dnsdumpster.com – The name says it all. Enter the target domain, hit search, profit!
- https://crt.sh/?q=%25target.com – Sometimes SSL is a goldmine of information. Use this site by searching for “%target.com” and it’ll get back with subdomains to you. Easy win.
- https://censys.io – Not greatbut has some useful information sometimes.
- http://searchdns.netcraft.com/ – Another to keep an eye on.
- https://www.shodan.io – Shodan is an infrastructure based spider with an associated information caching database that is made predominately for security professionals. It has historical and current data on a large swath of the internet’s servers, including seen-subdomains, server versioning, and much more.
Bruteforce :
Scraping what’s on the Internet can be a quick means to listing a good chunk of subdomains available on a target. The other, sometimes better, part can be found by testing common words as subdomains, sometimes even bruteforcing through the possible space of alphanumeric characters. If a non-guessable named subdomain is out there, bruteforcing can sometimes find it. And if you recursively bruteforce each subdomain found, you are on a good path to find assets that may have been forgotten about.
The most effective way to find subdomains via bruteforce is to type every possible subdomain in the browser and see if it resolves…
Just kidding! Joking aside, there are many tools that were either built specifically for dns bruteforce, or support such functionality. They make it easy to quickly iterate through a large wordlist and use threading for speed/efficiency. Here’s an inexhaustive list of tools that aid such endeavor:
- Subbrute – A DNS meta-query spider that enumerates DNS records, and subdomains.
- dnscan – a python wordlist-based DNS subdomain scanner.
- Nmap – Yes it’s a port scanner, but it can bruteforce subdomains too (check nmap scripts)
- Recon-Ng – The recon-ng framework has a brute_hosts module that allows to bruteforce subdomains.
- DNSRecon – A powerful DNS enumeration script
- Fierce – A semi-lightweight enumeration scanner
- Gobuster – Alternative directory and file busting tool written in Go
- DNSenum – Offers recursive and threaded subdomain enumeration.
- AltDNS – offers bruteforcing based on permutations of already found domains
This list is far from complete and many more awesome tools are out there.
Some of these tools have their built-in wordlists for bruteforcing, but others require you to specifically set it. There is a good (and big) list provided by Bitquark’s great research here. It is a list of 1000, 10000, 100000 and 1000000 most common subdomains found on the Internet. Depending on your available processing power, one of these lists will bring back solid results. The Seclists project also has concatenated and combined many of these tools lists for use with any subdomain bruteforcer, here (sorted_knock_dnsrecon_fierce_recon-ng.txt).
The following picture shows subbrute tool in action:
And nmap:
Putting it together
Jason Haddix, our Senior Director of Technical Operations, has written a quick script to automate the process of harvesting subdomains off public resources as well as bruteforcing them. The script, called Enumall.sh, is based on the Recon-Ng framework. Enumall.sh uses Google scraping, Bing scraping, Baidu scraping, Netcraft, and the SecLists project subdomain list. You can see a demo of the script here:
Google and some other modules of Recon-Ng require API keys to function properly. You can find instructions on how to do that in the Recon-ng Wiki.
This concludes the ultimate guide to finding subdomains on a target system. What to do with these results is pretty obvious…What are YOUR methods of subdomain discovery? Come discuss on the Bugcrowd Forum!
‘Til then,
Happy Hunting!
