Open Source Intelligence Techniques

OSINT Techniques: Chapter 16 - Domain Names & DNS

An exhaustive breakdown of Chapter 16 of Michael Bazzell's 'Open Source Intelligence Techniques', analyzing domain reconnaissance, historical WHOIS mining, Certificate Transparency log extraction, and advanced subdomain discovery pipelines.

8 min read
views
visits
OSINT Techniques Chapter 16 Summary - Domain Names & DNS
OSINT Techniques Chapter 16 Summary - Domain Names & DNS

Welcome to our sixteenth chapter analysis covering “Open Source Intelligence Techniques: Resources for Searching and Analyzing Online Information” (Sixth Edition) by Michael Bazzell.

Every website, API server, mail gateway, and corporate portal on the internet is anchored to a domain name. While the Domain Name System (DNS) was engineered to translate human-friendly domain names into machine-routable IP addresses, to the intelligence analyst and red team operative, DNS is a massive, publicly queryable hierarchical database detailing an organization’s infrastructure, cloud service providers, subsidiaries, and attack surfaces.

In Chapter 16: Domain Names & DNS, Bazzell outlines the technical discipline of domain reconnaissance. In the post-GDPR era, where standard WHOIS queries frequently return redacted placeholder strings (“Data Protected by Privacy”), investigators must master historical WHOIS repositories, DNS record parsing, Certificate Transparency (CT) log mining, and automated subdomain discovery pipelines. In this comprehensive guide, we dissect modern domain intelligence.


Key Takeaways

  • The Historical WHOIS Advantage: While modern WHOIS records are masked by privacy proxies or GDPR compliance, historical WHOIS records dating between 2000 and 2018 frequently preserve the original registrant’s real name, personal phone number, and physical billing address.
  • DNS Records as Infrastructure Leaks: Parsing A, AAAA, MX, TXT, and CNAME records reveals third-party software dependencies (Salesforce, Zendesk, Okta), mail validation tokens (SPF, DKIM, DMARC), and internal routing architectures.
  • Certificate Transparency (CT) Logs as the Ultimate Subdomain Source: Because modern web browsers require all public SSL/TLS certificates to be logged to publicly auditable Certificate Transparency logs, querying CT repositories (such as crt.sh) reveals newly created subdomains within seconds of certificate issuance.
  • Passive vs. Active Reconnaissance: True passive domain OSINT queries third-party aggregators (VirusTotal, SecurityTrails, AlienVault OTX) without sending a single DNS packet to the target organization’s authoritative nameservers, avoiding security operations center (SOC) detection.

Architectural & Investigative Workflow

The following flowchart outlines the multi-tiered domain reconnaissance and subdomain mapping methodology:

flowchart TD
    subgraph Target_Domain["Phase 1: Target Definition & Historical Registration"]
        Domain["Target Apex Domain (e.g., target-corp.com)"] --> WHOIS["Historical WHOIS Databases (DomainTools / WhoisXML)"]
        WHOIS --> Registrant["Unmask Historic Registrant Names, Phones, Emails"]
    end
    subgraph DNS_Record_Parsing["Phase 2: Authoritative DNS Interrogation"]
        Domain --> Dig["dig / dnsutils Record Extraction (A, MX, TXT, SOA)"]
        Dig --> SPF["TXT Record SPF Parsing (Authorized Egress Mail Servers)"]
        Dig --> MX_Rec["MX Routing & Vendor Identification (Proofpoint, Google Workspace)"]
    end
    subgraph Certificate_Transparency["Phase 3: Certificate Transparency & Passive Mapping"]
        Domain --> CrtSh["Certificate Transparency Logs (crt.sh API)"]
        Domain --> Subfinder["Subfinder & Amass Multi-Source Ingestion"]
        CrtSh --> SubdomainList["Discovered Subdomains (dev, staging, vpn, api)"]
        Subfinder --> SubdomainList
    end
    subgraph Attack_Surface_Synthesis["Phase 4: Attack Surface Dossier"]
        SubdomainList --> LiveProbe["HTTPx / DNS Resolution (Live vs Dead Assets)"]
        LiveProbe --> AttackSurfaceDossier["Complete Enterprise Attack Surface Map"]
    end

Technical Breakdown & Investigative Mechanics

1. Digging Deep: Authoritative DNS Parsing via CLI

Rather than relying on web portals, analysts utilize standard Unix DNS utilities like dig:

1
2
3
4
5
6
7
8
# Query all primary DNS record types simultaneously
dig target-corp.com ANY +noall +answer

# Query specific authoritative Name Servers (NS)
dig target-corp.com NS +short

# Query Mail Exchanger (MX) records
dig target-corp.com MX +short

Mining TXT Records for Third-Party Vendor Footprints:

The DNS TXT record is frequently abused to store domain ownership verification strings:

1
dig target-corp.com TXT +short

Sample Real-World TXT Output:

1
2
3
4
5
"v=spf1 include:_spf.google.com include:servers.mcsv.net include:mailgun.org ip4:198.51.100.0/24 -all"
"google-site-verification=rX9kL2z8Jq7..."
"atlassian-domain-verification=3kL9p0z..."
"docusign=059a42e1..."
"adobe-id-verification=189a71b..."
  • Intelligence Gained: The target utilizes Google Workspace for corporate email, Mailchimp and Mailgun for marketing, Atlassian Cloud (Jira/Confluence), DocuSign for legal contracts, and Adobe Creative Cloud.

2. Mining Certificate Transparency Logs via crt.sh

Certificate Transparency (CT) is an open framework designed to detect misissued SSL certificates. Every time an enterprise requests an SSL certificate (even for an internal staging or testing server), it is permanently logged to public CT ledgers.

1
2
3
4
5
6
7
8
9
#!/usr/bin/env bash
# crtsh_subdomain_miner.sh - Automated Subdomain Extraction from CT Logs
TARGET_DOMAIN="target-corp.com"
echo "[*] Querying crt.sh for %.$TARGET_DOMAIN..."

curl -s "https://crt.sh/?q=%25.${TARGET_DOMAIN}&output=json"     | jq -r '.[].name_value'     | sed 's/\*\.//g'     | sort -u > "ct_subdomains_${TARGET_DOMAIN}.txt"

echo "[+] Discovered $(wc -l < "ct_subdomains_${TARGET_DOMAIN}.txt") unique subdomains."
head -n 10 "ct_subdomains_${TARGET_DOMAIN}.txt"

High-Value Subdomains Frequently Discovered via CT Logs:

  • vpn.target-corp.com (Exposed SSL-VPN gateways: Fortinet, Pulse Secure, Cisco AnyConnect)
  • jenkins-staging.target-corp.com (Unauthenticated CI/CD build pipelines)
  • grafana.target-corp.com (Internal metric dashboards)
  • dev-api-v2.target-corp.com (Staging API endpoints with weak authentication)

3. Automated Subdomain Reconnaissance Pipeline

Modern offensive security teams combine multiple passive data sources into a unified pipeline using ProjectDiscovery’s Subfinder and HTTPx:

1
2
3
4
5
# Step 1: Execute passive subdomain discovery across 30+ free API sources
subfinder -d target-corp.com -silent -all -o subdomains_raw.txt

# Step 2: Probe discovered subdomains for active HTTP/HTTPS web servers
httpx -l subdomains_raw.txt       -silent       -status-code       -title       -tech-detect       -follow-redirects       -o live_web_assets.txt

Output Analysis:

1
2
3
4
https://vpn.target-corp.com [200] [Pulse Connect Secure SSL VPN] [Pulse Secure]
https://wiki.target-corp.com [200] [Confluence - Acme Corp] [Atlassian Confluence, Java]
https://api.target-corp.com [401] [Unauthorized API Gateway] [Nginx, Express]
https://test.target-corp.com [403] [403 Forbidden] [Apache 2.4.41]

4. Testing for DNS Zone Transfers (AXFR)

A DNS Zone Transfer (AXFR) is a mechanism intended to replicate DNS databases between primary and secondary nameservers. If misconfigured, an unauthenticated external user can download the entire DNS zone file in a single query:

1
2
3
4
5
6
7
8
# Identify authoritative nameservers
NAMESERVERS=$(dig target-corp.com NS +short)

# Attempt zone transfer against each nameserver
for ns in $NAMESERVERS; do
    echo "[*] Testing Zone Transfer against: $ns"
    dig axfr @$ns target-corp.com
done
  • If the server responds with a complete dump of internal hostnames and IP addresses, the domain suffers from a critical DNS misconfiguration.

Real-World Case Scenario

Case Study: Red Team Perimeter Infiltration via Orphaned Subdomain

During an external adversary emulation engagement against an international retailer, the client’s primary web application (www.client-retail.com) was protected by a strict Cloudflare enterprise WAF.

The red team initiated domain reconnaissance:

  1. Querying crt.sh revealed a certificate issued three weeks prior for: dev-payments-gateway.client-retail.com.
  2. Resolving the IP address of this subdomain showed it pointed directly to an unshielded AWS Elastic IP address, bypassing Cloudflare completely.
  3. Visiting the subdomain presented an unfinished developer payment portal with default Django debug mode enabled (DEBUG = True).
  4. Triggering a synthetic HTTP error revealed the server’s internal environment variables, including AWS access keys and database credentials, resulting in full perimeter breach.

Defensive Countermeasures & OpSec Recommendations

For network architects, DNS administrators, and corporate security teams:

  1. Disable Unrestricted DNS Zone Transfers: Ensure all authoritative nameservers explicitly deny AXFR queries from unauthorized IP addresses (allow-transfer { none; }; in BIND).
  2. Monitor Certificate Transparency Logs: Deploy automated monitoring (via Certstream or Cloudflare CT Monitoring) to receive real-time alerts whenever a new SSL certificate is issued for your enterprise domain.
  3. Decommission Orphaned DNS Records: Regularly audit DNS CNAME records pointing to external SaaS providers (AWS S3, GitHub Pages, Zendesk) to prevent Subdomain Takeover attacks.

Conclusion & Next Steps

Chapter 16 proves that domain reconnaissance is the bedrock of attack surface management. By leveraging authoritative DNS queries, historical WHOIS archives, and Certificate Transparency log mining, an investigator can map an enterprise footprint comprehensively.

In Chapter 17: IP Addresses & Networks, we advance from domain names to raw network infrastructure: BGP routing tables, ASN mapping, passive Shodan scanning, and SSL JARM/JA3 fingerprinting.


Practical Domain Name & Passive DNS Intelligence Cheat-Sheet

Enumeration Vector Command / Tool Syntax Intelligence Objective
Authoritative WHOIS Query whois -h whois.iana.org target.com Identifies registrar, creation dates, DNSSEC status, and nameserver delegation
Certificate Transparency Mining curl -s 'https://crt.sh/?q=%25.target.com&output=json' | jq -r '.[].name_value' | sort -u Discovers hidden subdomains and internal staging hosts via public SSL logs
Comprehensive Zone Check dig +nocmd target.com ANY +multiline +noall +answer Pulls active A, AAAA, MX, TXT, SOA, and CNAME resource records
Subdomain Passive DNS Scan sublist3r -d target.com -o subdomains.txt Aggregates passive DNS records from Netcraft, Virustotal, ThreatCrowd, and SSL certs
Historical Reverse IP Triage securitytrails.com / viewdns.info reverse ip Maps co-hosted virtual hosts sharing the same historical IP address space

Authoritative References & Verified Investigative Sources


Author: Abdulrahman M. Fathi (0x3bdo) • Cyber Security Engineer & Penetration Tester

Open Research & Community Labs
صدقة جارية • Ongoing Charity

Empowering the Cybersecurity Community

All research, lab walkthroughs, and technical series on this platform remain 100% free. You can support server hosting, research environments, and lab infrastructure.

Support & Wall of Honor
اللهم اجعل هذا العمل علمًا نافعًا وصدقةً جارية، واكتب لي به الأجر والقبول، وانفع به كل من قرأه أو تعلّمه.
اللهم احفظ لي والديّ وأهلي، وبارك في أعمارهم وصحتهم، وأدم عليهم نعمة الصحة والعافية، ولا تريني فيهم بأسًا يبكيني.
Dedicated as ongoing charity and beneficial knowledge. May Allah bless my parents and family, protect them, and grant them lasting health and happiness.