OSINT Techniques: Chapter 22 - Recon-ng
An exhaustive breakdown of Chapter 22 of Michael Bazzell's 'Open Source Intelligence Techniques', analyzing the Recon-ng v5 framework architecture, database workspace schemas, module chaining, and automated intelligence dossier generation.
Welcome to our twenty-second chapter analysis covering “Open Source Intelligence Techniques: Resources for Searching and Analyzing Online Information” (Sixth Edition) by Michael Bazzell.
When conducting large-scale investigations, an analyst quickly discovers that the bottleneck is not finding data, but organizing, normalizing, and correlating data. Copying and pasting hundreds of domains, IP addresses, subdomains, email addresses, and employee names into disconnected text files or spreadsheets leads to missed correlations and lost evidence.
In Chapter 22: Recon-ng, Bazzell examines the premier open-source automated reconnaissance framework: Recon-ng. Architected by Tim Tomes (lanmaster53), Recon-ng provides a modular, command-line interface modeled after Metasploit. It features a structured relational database backend that automatically normalizes discovered entities, allowing analysts to chain reconnaissance modules together into an automated pipeline. In this comprehensive guide, we dissect Recon-ng v5 workspace architecture, database schemas, module chaining, and automated dossier reporting.
Key Takeaways
- The Relational Workspace Model: Recon-ng organizes cases into isolated workspaces powered by a local SQLite database schema. Data discovered by one module (e.g., a domain) automatically populates the input queue for the next module (e.g., subdomain bruteforcing or email harvesting).
- The Metasploit Paradigm for OSINT: Recon-ng eliminates custom scripting overhead by standardizing commands (
workspaces,modules load,options set,run), creating a consistent operational workflow for analysts. - Module Chaining Workflows: Modules are categorized by input and output types (
recon/domains-hosts/,recon/hosts-hosts/,recon/contacts-profiles/). Chaining these modules enables automated multi-stage reconnaissance from a single seed domain. - API Key Management: Recon-ng centralizes third-party API keys (Shodan, Censys, HaveIBeenPwned, Hunter.io, WhoisXML) in a global configuration store, allowing modules to leverage premium data sources seamlessly.
- Automated Evidence Reporting: The framework includes built-in reporting modules (
reporting/html,reporting/csv) that automatically compile tables of hosts, contacts, and vulnerabilities into presentation-ready executive briefs.
Architectural & Investigative Workflow
The following flowchart outlines the Recon-ng database schema and modular chaining pipeline:
flowchart TD
subgraph Workspace_Initialization["Phase 1: Workspace & Seed Ingestion"]
Init["workspaces create TargetCorp"] --> AddDomain["db insert domains (target-corp.com)"]
AddDomain --> DB_Domains[("Database Table: domains")]
end
subgraph Module_Chaining["Phase 2: Modular Reconnaissance Chaining"]
DB_Domains --> Mod1["modules load recon/domains-hosts/sublist3r"]
Mod1 --> DB_Hosts[("Database Table: hosts")]
DB_Hosts --> Mod2["modules load recon/hosts-hosts/resolve"]
Mod2 --> DB_Hosts_Resolved[("Database Table: hosts (IPs Resolved)")]
DB_Domains --> Mod3["modules load recon/domains-contacts/hunter_io"]
Mod3 --> DB_Contacts[("Database Table: contacts (Names & Emails)")]
end
subgraph Vulnerability_Discovery["Phase 3: Vulnerability & Threat Mapping"]
DB_Hosts_Resolved --> Mod4["modules load recon/hosts-hosts/shodan_hostname"]
Mod4 --> DB_Ports[("Database Table: ports & vulnerabilities")]
end
subgraph Automated_Reporting["Phase 4: Synthesis & Reporting"]
DB_Hosts_Resolved --> ReportEngine["modules load reporting/html"]
DB_Contacts --> ReportEngine
DB_Ports --> ReportEngine
ReportEngine --> FinalReport["Executive HTML Intelligence Dossier"]
end
Technical Breakdown & Investigative Mechanics
1. Recon-ng Workspace & Database Schema Mechanics
Recon-ng is structured around an internal SQLite database with standardized tables:
1
2
3
4
5
6
7
8
9
Standard Recon-ng Database Tables:
- domains: Primary target domain names.
- companies: Target corporate organization names.
- netblocks: Discovered IP address CIDR ranges.
- locations: Physical geographic addresses.
- hosts: Resolved hostnames, subdomains, and IP addresses.
- contacts: Harvested human names, email addresses, and roles.
- credentials: Leaked cleartext passwords and hashes.
- vulnerabilities: Discovered CVEs and open server misconfigurations.
Step-by-Step CLI Workspace Setup:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Launch Recon-ng CLI
recon-ng
# Create and select a dedicated workspace for the investigation
[recon-ng][default] > workspaces create TargetCorp_Investigation
[recon-ng][TargetCorp_Investigation] >
# Add initial seed targets into the database
[recon-ng][TargetCorp_Investigation] > db insert domains
domain (TEXT): target-corp.com
notes (TEXT): Primary apex corporate domain
# Verify inserted record in the internal database
[recon-ng][TargetCorp_Investigation] > show domains
2. Marketplace & Modular Engine Management
Recon-ng v5 separates the core framework from modules. Analysts install modules from an official marketplace:
1
2
3
4
5
6
7
8
9
# Search the Recon-ng marketplace for subdomain and host discovery modules
[recon-ng][TargetCorp_Investigation] > marketplace search hosts
# Install essential modules
[recon-ng][TargetCorp_Investigation] > marketplace install recon/domains-hosts/brute_hosts
[recon-ng][TargetCorp_Investigation] > marketplace install recon/domains-hosts/certificate_transparency
[recon-ng][TargetCorp_Investigation] > marketplace install recon/hosts-hosts/resolve
[recon-ng][TargetCorp_Investigation] > marketplace install recon/domains-contacts/hunter_io
[recon-ng][TargetCorp_Investigation] > marketplace install reporting/html
Configuring Global Third-Party API Keys:
1
2
3
[recon-ng][TargetCorp_Investigation] > keys add hunter_io a9182374b0192834
[recon-ng][TargetCorp_Investigation] > keys add shodan_api 0192834019283412
[recon-ng][TargetCorp_Investigation] > keys list
3. Executing a Chained Reconnaissance Workflow
The true power of Recon-ng is automated pipeline execution. Discovered data flows organically between tables:
Step 1: Harvest Subdomains via Certificate Transparency
1
2
3
4
[recon-ng][TargetCorp_Investigation] > modules load recon/domains-hosts/certificate_transparency
[recon-ng][TargetCorp_Investigation][certificate_transparency] > run
[*] URL: https://crt.sh/?q=%25.target-corp.com&output=json
[*] 42 hosts added to the database.
Step 2: Resolve Discovered Subdomains to IP Addresses
1
2
3
4
5
6
7
# Load the resolver module which reads un-resolved hosts from the hosts table
[recon-ng][TargetCorp_Investigation] > modules load recon/hosts-hosts/resolve
[recon-ng][TargetCorp_Investigation][resolve] > run
[*] Resolving hostnames...
[*] api.target-corp.com resolved to 198.51.100.25
[*] vpn.target-corp.com resolved to 198.51.100.44
[*] 38 host records updated.
Step 3: Harvest Corporate Employee Contacts
1
2
3
4
[recon-ng][TargetCorp_Investigation] > modules load recon/domains-contacts/hunter_io
[recon-ng][TargetCorp_Investigation][hunter_io] > run
[*] Harvesting contacts for domain target-corp.com...
[*] 18 contacts added to the database.
4. Automated Report Generation
Once reconnaissance modules complete, compile all database tables into an executive dossier:
1
2
3
4
5
6
7
[recon-ng][TargetCorp_Investigation] > modules load reporting/html
[recon-ng][TargetCorp_Investigation][html] > options set CREATOR Abdulrahman M. Fathi
[recon-ng][TargetCorp_Investigation][html] > options set CUSTOMER Target Corp Executive Board
[recon-ng][TargetCorp_Investigation][html] > options set FILENAME /home/analyst/Reports/TargetCorp_OSINT.html
[recon-ng][TargetCorp_Investigation][html] > run
[*] Generating report...
[*] Report generated: /home/analyst/Reports/TargetCorp_OSINT.html
- Opening this HTML file presents interactive, sortable tables detailing every host, IP, contact, and vulnerability discovered during the engagement.
Real-World Case Scenario
Case Study: Red Team Pre-Attack Surface Mapping in Under 30 Minutes
An offensive security firm was contracted to conduct a blind red team penetration test against a regional hospital system with over 200 physical clinics.
The lead intelligence analyst deployed Recon-ng:
- Ingested the hospital’s primary domain into a fresh workspace.
- Chained
certificate_transparencyandbrute_hoststo discover 145 unique subdomains.- Ran
hosts-hosts/resolveto map all subdomains to IP blocks.- Ran
hosts-hosts/shodan_hostnameto query exposed services across all resolved IPs out-of-band.- Within 25 minutes, Recon-ng’s database flagged an unauthenticated DICOM medical imaging PACS server (
pacs-archive.hospital.org) exposed directly on port 104 with public directory browsing enabled.- The finding was documented in an automated HTML report and escalated immediately to hospital leadership for remediation before adversarial compromise.
Defensive Countermeasures & OpSec Recommendations
For corporate enterprise defenders and SOC managers:
- Conduct Recon-ng Self-Audits: Periodically execute Recon-ng sweeps against your own corporate domain names to discover what external threat actors can see in under 15 minutes.
- Scrub Public Contact Harvesters: Submit removal requests to data scrapers like Hunter.io and RocketReach to suppress corporate email naming conventions.
- Blacklist Scanning Engines at the Edge: Block known Shodan, Censys, and Project Sonar scanner IP ranges at the border firewall to prevent automated passive banner aggregation.
Conclusion & Next Steps
Chapter 22 proves that automation and relational database management transform scattered OSINT findings into cohesive, actionable intelligence. By standardizing module execution and chaining data flows, an investigator operates with maximum speed and consistency.
In Chapter 23: Radio Frequency & SIGINT, we expand beyond internet networks into open-source Signals Intelligence: Software Defined Radios (RTL-SDR), ADS-B aircraft flight decoding, and maritime AIS vessel tracking.
Practical Recon-ng Modular Intelligence Automation Cheat-Sheet
| Recon-ng Command | Operational Syntax | Intelligence Function |
|---|---|---|
| Initialize Case Workspace | workspaces create Case_TargetInvestigation |
Builds an isolated relational SQLite database container for current engagement |
| Add Target Seeds | db insert domains targetcorp.com |
Injects initial domain or entity seed values into the reconnaissance database |
| Execute Modular Worker | modules load recon/domains-hosts/certificate_transparency && run |
Harvests subdomains via SSL transparency logs and automatically stores them in db |
| Enrich Discovered Hosts | modules load recon/hosts-hosts/resolve && run |
Resolves discovered subdomains to active IP addresses using multi-threaded DNS |
| Case Intelligence Summary | dashboard && show hosts |
Outputs interactive visual metrics and structured table of all discovered assets |
Authoritative References & Verified Investigative Sources
- Michael Bazzell, IntelTechniques: Automated Reconnaissance Frameworks and Recon-ng. https://inteltechniques.com/book1.html.
- Recon-ng Framework: Tim Tomes (lanmaster53) Official Recon-ng Repository. https://github.com/lanmaster53/recon-ng.
- SANS Institute: Mastering Recon-ng for Modern Open-Source Intelligence. https://www.sans.org/.
- Black Hat Briefings: Recon-ng: Next Generation Web Reconnaissance. https://www.blackhat.com/.
- NIST SP 800-115: Technical Guide to Information Security Testing and Assessment. https://csrc.nist.gov/publications/detail/sp/800-115/final.
Author: Abdulrahman M. Fathi (0x3bdo) • Cyber Security Engineer & Penetration Tester