OSINT Techniques: Chapter 16 - Domain Names & DNS
An exhaustive breakdown of Chapter 16 of Michael Bazzell's 'Open Source Intelligence Techniques', analyzing domain reconnaissance, historical WHOIS mining, Certificate Transparency log extraction, and advanced subdomain discovery pipelines.
Welcome to our sixteenth chapter analysis covering “Open Source Intelligence Techniques: Resources for Searching and Analyzing Online Information” (Sixth Edition) by Michael Bazzell.
Every website, API server, mail gateway, and corporate portal on the internet is anchored to a domain name. While the Domain Name System (DNS) was engineered to translate human-friendly domain names into machine-routable IP addresses, to the intelligence analyst and red team operative, DNS is a massive, publicly queryable hierarchical database detailing an organization’s infrastructure, cloud service providers, subsidiaries, and attack surfaces.
In Chapter 16: Domain Names & DNS, Bazzell outlines the technical discipline of domain reconnaissance. In the post-GDPR era, where standard WHOIS queries frequently return redacted placeholder strings (“Data Protected by Privacy”), investigators must master historical WHOIS repositories, DNS record parsing, Certificate Transparency (CT) log mining, and automated subdomain discovery pipelines. In this comprehensive guide, we dissect modern domain intelligence.
Key Takeaways
- The Historical WHOIS Advantage: While modern WHOIS records are masked by privacy proxies or GDPR compliance, historical WHOIS records dating between 2000 and 2018 frequently preserve the original registrant’s real name, personal phone number, and physical billing address.
- DNS Records as Infrastructure Leaks: Parsing A, AAAA, MX, TXT, and CNAME records reveals third-party software dependencies (Salesforce, Zendesk, Okta), mail validation tokens (SPF, DKIM, DMARC), and internal routing architectures.
- Certificate Transparency (CT) Logs as the Ultimate Subdomain Source: Because modern web browsers require all public SSL/TLS certificates to be logged to publicly auditable Certificate Transparency logs, querying CT repositories (such as
crt.sh) reveals newly created subdomains within seconds of certificate issuance. - Passive vs. Active Reconnaissance: True passive domain OSINT queries third-party aggregators (VirusTotal, SecurityTrails, AlienVault OTX) without sending a single DNS packet to the target organization’s authoritative nameservers, avoiding security operations center (SOC) detection.
Architectural & Investigative Workflow
The following flowchart outlines the multi-tiered domain reconnaissance and subdomain mapping methodology:
flowchart TD
subgraph Target_Domain["Phase 1: Target Definition & Historical Registration"]
Domain["Target Apex Domain (e.g., target-corp.com)"] --> WHOIS["Historical WHOIS Databases (DomainTools / WhoisXML)"]
WHOIS --> Registrant["Unmask Historic Registrant Names, Phones, Emails"]
end
subgraph DNS_Record_Parsing["Phase 2: Authoritative DNS Interrogation"]
Domain --> Dig["dig / dnsutils Record Extraction (A, MX, TXT, SOA)"]
Dig --> SPF["TXT Record SPF Parsing (Authorized Egress Mail Servers)"]
Dig --> MX_Rec["MX Routing & Vendor Identification (Proofpoint, Google Workspace)"]
end
subgraph Certificate_Transparency["Phase 3: Certificate Transparency & Passive Mapping"]
Domain --> CrtSh["Certificate Transparency Logs (crt.sh API)"]
Domain --> Subfinder["Subfinder & Amass Multi-Source Ingestion"]
CrtSh --> SubdomainList["Discovered Subdomains (dev, staging, vpn, api)"]
Subfinder --> SubdomainList
end
subgraph Attack_Surface_Synthesis["Phase 4: Attack Surface Dossier"]
SubdomainList --> LiveProbe["HTTPx / DNS Resolution (Live vs Dead Assets)"]
LiveProbe --> AttackSurfaceDossier["Complete Enterprise Attack Surface Map"]
end
Technical Breakdown & Investigative Mechanics
1. Digging Deep: Authoritative DNS Parsing via CLI
Rather than relying on web portals, analysts utilize standard Unix DNS utilities like dig:
1
2
3
4
5
6
7
8
# Query all primary DNS record types simultaneously
dig target-corp.com ANY +noall +answer
# Query specific authoritative Name Servers (NS)
dig target-corp.com NS +short
# Query Mail Exchanger (MX) records
dig target-corp.com MX +short
Mining TXT Records for Third-Party Vendor Footprints:
The DNS TXT record is frequently abused to store domain ownership verification strings:
1
dig target-corp.com TXT +short
Sample Real-World TXT Output:
1
2
3
4
5
"v=spf1 include:_spf.google.com include:servers.mcsv.net include:mailgun.org ip4:198.51.100.0/24 -all"
"google-site-verification=rX9kL2z8Jq7..."
"atlassian-domain-verification=3kL9p0z..."
"docusign=059a42e1..."
"adobe-id-verification=189a71b..."
- Intelligence Gained: The target utilizes Google Workspace for corporate email, Mailchimp and Mailgun for marketing, Atlassian Cloud (Jira/Confluence), DocuSign for legal contracts, and Adobe Creative Cloud.
2. Mining Certificate Transparency Logs via crt.sh
Certificate Transparency (CT) is an open framework designed to detect misissued SSL certificates. Every time an enterprise requests an SSL certificate (even for an internal staging or testing server), it is permanently logged to public CT ledgers.
1
2
3
4
5
6
7
8
9
#!/usr/bin/env bash
# crtsh_subdomain_miner.sh - Automated Subdomain Extraction from CT Logs
TARGET_DOMAIN="target-corp.com"
echo "[*] Querying crt.sh for %.$TARGET_DOMAIN..."
curl -s "https://crt.sh/?q=%25.${TARGET_DOMAIN}&output=json" | jq -r '.[].name_value' | sed 's/\*\.//g' | sort -u > "ct_subdomains_${TARGET_DOMAIN}.txt"
echo "[+] Discovered $(wc -l < "ct_subdomains_${TARGET_DOMAIN}.txt") unique subdomains."
head -n 10 "ct_subdomains_${TARGET_DOMAIN}.txt"
High-Value Subdomains Frequently Discovered via CT Logs:
vpn.target-corp.com(Exposed SSL-VPN gateways: Fortinet, Pulse Secure, Cisco AnyConnect)jenkins-staging.target-corp.com(Unauthenticated CI/CD build pipelines)grafana.target-corp.com(Internal metric dashboards)dev-api-v2.target-corp.com(Staging API endpoints with weak authentication)
3. Automated Subdomain Reconnaissance Pipeline
Modern offensive security teams combine multiple passive data sources into a unified pipeline using ProjectDiscovery’s Subfinder and HTTPx:
1
2
3
4
5
# Step 1: Execute passive subdomain discovery across 30+ free API sources
subfinder -d target-corp.com -silent -all -o subdomains_raw.txt
# Step 2: Probe discovered subdomains for active HTTP/HTTPS web servers
httpx -l subdomains_raw.txt -silent -status-code -title -tech-detect -follow-redirects -o live_web_assets.txt
Output Analysis:
1
2
3
4
https://vpn.target-corp.com [200] [Pulse Connect Secure SSL VPN] [Pulse Secure]
https://wiki.target-corp.com [200] [Confluence - Acme Corp] [Atlassian Confluence, Java]
https://api.target-corp.com [401] [Unauthorized API Gateway] [Nginx, Express]
https://test.target-corp.com [403] [403 Forbidden] [Apache 2.4.41]
4. Testing for DNS Zone Transfers (AXFR)
A DNS Zone Transfer (AXFR) is a mechanism intended to replicate DNS databases between primary and secondary nameservers. If misconfigured, an unauthenticated external user can download the entire DNS zone file in a single query:
1
2
3
4
5
6
7
8
# Identify authoritative nameservers
NAMESERVERS=$(dig target-corp.com NS +short)
# Attempt zone transfer against each nameserver
for ns in $NAMESERVERS; do
echo "[*] Testing Zone Transfer against: $ns"
dig axfr @$ns target-corp.com
done
- If the server responds with a complete dump of internal hostnames and IP addresses, the domain suffers from a critical DNS misconfiguration.
Real-World Case Scenario
Case Study: Red Team Perimeter Infiltration via Orphaned Subdomain
During an external adversary emulation engagement against an international retailer, the client’s primary web application (
www.client-retail.com) was protected by a strict Cloudflare enterprise WAF.The red team initiated domain reconnaissance:
- Querying
crt.shrevealed a certificate issued three weeks prior for:dev-payments-gateway.client-retail.com.- Resolving the IP address of this subdomain showed it pointed directly to an unshielded AWS Elastic IP address, bypassing Cloudflare completely.
- Visiting the subdomain presented an unfinished developer payment portal with default Django debug mode enabled (
DEBUG = True).- Triggering a synthetic HTTP error revealed the server’s internal environment variables, including AWS access keys and database credentials, resulting in full perimeter breach.
Defensive Countermeasures & OpSec Recommendations
For network architects, DNS administrators, and corporate security teams:
- Disable Unrestricted DNS Zone Transfers: Ensure all authoritative nameservers explicitly deny
AXFRqueries from unauthorized IP addresses (allow-transfer { none; };in BIND). - Monitor Certificate Transparency Logs: Deploy automated monitoring (via Certstream or Cloudflare CT Monitoring) to receive real-time alerts whenever a new SSL certificate is issued for your enterprise domain.
- Decommission Orphaned DNS Records: Regularly audit DNS CNAME records pointing to external SaaS providers (AWS S3, GitHub Pages, Zendesk) to prevent Subdomain Takeover attacks.
Conclusion & Next Steps
Chapter 16 proves that domain reconnaissance is the bedrock of attack surface management. By leveraging authoritative DNS queries, historical WHOIS archives, and Certificate Transparency log mining, an investigator can map an enterprise footprint comprehensively.
In Chapter 17: IP Addresses & Networks, we advance from domain names to raw network infrastructure: BGP routing tables, ASN mapping, passive Shodan scanning, and SSL JARM/JA3 fingerprinting.
Practical Domain Name & Passive DNS Intelligence Cheat-Sheet
| Enumeration Vector | Command / Tool Syntax | Intelligence Objective |
|---|---|---|
| Authoritative WHOIS Query | whois -h whois.iana.org target.com |
Identifies registrar, creation dates, DNSSEC status, and nameserver delegation |
| Certificate Transparency Mining | curl -s 'https://crt.sh/?q=%25.target.com&output=json' | jq -r '.[].name_value' | sort -u |
Discovers hidden subdomains and internal staging hosts via public SSL logs |
| Comprehensive Zone Check | dig +nocmd target.com ANY +multiline +noall +answer |
Pulls active A, AAAA, MX, TXT, SOA, and CNAME resource records |
| Subdomain Passive DNS Scan | sublist3r -d target.com -o subdomains.txt |
Aggregates passive DNS records from Netcraft, Virustotal, ThreatCrowd, and SSL certs |
| Historical Reverse IP Triage | securitytrails.com / viewdns.info reverse ip |
Maps co-hosted virtual hosts sharing the same historical IP address space |
Authoritative References & Verified Investigative Sources
- Michael Bazzell, IntelTechniques: Domain Names, DNS Forensics and Network Records. https://inteltechniques.com/book1.html.
- crt.sh (Sectigo): Certificate Transparency Log Search Engine. https://crt.sh/.
- IANA Registries: Root Zone Database & TLD Information. https://www.iana.org/domains/root/db.
- SecurityTrails Research: Passive DNS & Domain Historical Data Architecture. https://securitytrails.com/.
- IETF RFC 1035: Domain Names - Implementation and Specification. https://datatracker.ietf.org/doc/html/rfc1035.
Author: Abdulrahman M. Fathi (0x3bdo) • Cyber Security Engineer & Penetration Tester