Open Source Intelligence Techniques

OSINT Techniques: Chapter 22 - Recon-ng

An exhaustive breakdown of Chapter 22 of Michael Bazzell's 'Open Source Intelligence Techniques', analyzing the Recon-ng v5 framework architecture, database workspace schemas, module chaining, and automated intelligence dossier generation.

8 min read
views
visits
OSINT Techniques Chapter 22 Summary - Recon-ng
OSINT Techniques Chapter 22 Summary - Recon-ng

Welcome to our twenty-second chapter analysis covering “Open Source Intelligence Techniques: Resources for Searching and Analyzing Online Information” (Sixth Edition) by Michael Bazzell.

When conducting large-scale investigations, an analyst quickly discovers that the bottleneck is not finding data, but organizing, normalizing, and correlating data. Copying and pasting hundreds of domains, IP addresses, subdomains, email addresses, and employee names into disconnected text files or spreadsheets leads to missed correlations and lost evidence.

In Chapter 22: Recon-ng, Bazzell examines the premier open-source automated reconnaissance framework: Recon-ng. Architected by Tim Tomes (lanmaster53), Recon-ng provides a modular, command-line interface modeled after Metasploit. It features a structured relational database backend that automatically normalizes discovered entities, allowing analysts to chain reconnaissance modules together into an automated pipeline. In this comprehensive guide, we dissect Recon-ng v5 workspace architecture, database schemas, module chaining, and automated dossier reporting.


Key Takeaways

  • The Relational Workspace Model: Recon-ng organizes cases into isolated workspaces powered by a local SQLite database schema. Data discovered by one module (e.g., a domain) automatically populates the input queue for the next module (e.g., subdomain bruteforcing or email harvesting).
  • The Metasploit Paradigm for OSINT: Recon-ng eliminates custom scripting overhead by standardizing commands (workspaces, modules load, options set, run), creating a consistent operational workflow for analysts.
  • Module Chaining Workflows: Modules are categorized by input and output types (recon/domains-hosts/, recon/hosts-hosts/, recon/contacts-profiles/). Chaining these modules enables automated multi-stage reconnaissance from a single seed domain.
  • API Key Management: Recon-ng centralizes third-party API keys (Shodan, Censys, HaveIBeenPwned, Hunter.io, WhoisXML) in a global configuration store, allowing modules to leverage premium data sources seamlessly.
  • Automated Evidence Reporting: The framework includes built-in reporting modules (reporting/html, reporting/csv) that automatically compile tables of hosts, contacts, and vulnerabilities into presentation-ready executive briefs.

Architectural & Investigative Workflow

The following flowchart outlines the Recon-ng database schema and modular chaining pipeline:

flowchart TD
    subgraph Workspace_Initialization["Phase 1: Workspace & Seed Ingestion"]
        Init["workspaces create TargetCorp"] --> AddDomain["db insert domains (target-corp.com)"]
        AddDomain --> DB_Domains[("Database Table: domains")]
    end
    subgraph Module_Chaining["Phase 2: Modular Reconnaissance Chaining"]
        DB_Domains --> Mod1["modules load recon/domains-hosts/sublist3r"]
        Mod1 --> DB_Hosts[("Database Table: hosts")]
        DB_Hosts --> Mod2["modules load recon/hosts-hosts/resolve"]
        Mod2 --> DB_Hosts_Resolved[("Database Table: hosts (IPs Resolved)")]
        DB_Domains --> Mod3["modules load recon/domains-contacts/hunter_io"]
        Mod3 --> DB_Contacts[("Database Table: contacts (Names & Emails)")]
    end
    subgraph Vulnerability_Discovery["Phase 3: Vulnerability & Threat Mapping"]
        DB_Hosts_Resolved --> Mod4["modules load recon/hosts-hosts/shodan_hostname"]
        Mod4 --> DB_Ports[("Database Table: ports & vulnerabilities")]
    end
    subgraph Automated_Reporting["Phase 4: Synthesis & Reporting"]
        DB_Hosts_Resolved --> ReportEngine["modules load reporting/html"]
        DB_Contacts --> ReportEngine
        DB_Ports --> ReportEngine
        ReportEngine --> FinalReport["Executive HTML Intelligence Dossier"]
    end

Technical Breakdown & Investigative Mechanics

1. Recon-ng Workspace & Database Schema Mechanics

Recon-ng is structured around an internal SQLite database with standardized tables:

1
2
3
4
5
6
7
8
9
Standard Recon-ng Database Tables:
- domains:      Primary target domain names.
- companies:    Target corporate organization names.
- netblocks:    Discovered IP address CIDR ranges.
- locations:    Physical geographic addresses.
- hosts:        Resolved hostnames, subdomains, and IP addresses.
- contacts:     Harvested human names, email addresses, and roles.
- credentials:  Leaked cleartext passwords and hashes.
- vulnerabilities: Discovered CVEs and open server misconfigurations.

Step-by-Step CLI Workspace Setup:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Launch Recon-ng CLI
recon-ng

# Create and select a dedicated workspace for the investigation
[recon-ng][default] > workspaces create TargetCorp_Investigation
[recon-ng][TargetCorp_Investigation] > 

# Add initial seed targets into the database
[recon-ng][TargetCorp_Investigation] > db insert domains
domain (TEXT): target-corp.com
notes (TEXT): Primary apex corporate domain

# Verify inserted record in the internal database
[recon-ng][TargetCorp_Investigation] > show domains

2. Marketplace & Modular Engine Management

Recon-ng v5 separates the core framework from modules. Analysts install modules from an official marketplace:

1
2
3
4
5
6
7
8
9
# Search the Recon-ng marketplace for subdomain and host discovery modules
[recon-ng][TargetCorp_Investigation] > marketplace search hosts

# Install essential modules
[recon-ng][TargetCorp_Investigation] > marketplace install recon/domains-hosts/brute_hosts
[recon-ng][TargetCorp_Investigation] > marketplace install recon/domains-hosts/certificate_transparency
[recon-ng][TargetCorp_Investigation] > marketplace install recon/hosts-hosts/resolve
[recon-ng][TargetCorp_Investigation] > marketplace install recon/domains-contacts/hunter_io
[recon-ng][TargetCorp_Investigation] > marketplace install reporting/html

Configuring Global Third-Party API Keys:

1
2
3
[recon-ng][TargetCorp_Investigation] > keys add hunter_io a9182374b0192834
[recon-ng][TargetCorp_Investigation] > keys add shodan_api 0192834019283412
[recon-ng][TargetCorp_Investigation] > keys list

3. Executing a Chained Reconnaissance Workflow

The true power of Recon-ng is automated pipeline execution. Discovered data flows organically between tables:

Step 1: Harvest Subdomains via Certificate Transparency

1
2
3
4
[recon-ng][TargetCorp_Investigation] > modules load recon/domains-hosts/certificate_transparency
[recon-ng][TargetCorp_Investigation][certificate_transparency] > run
[*] URL: https://crt.sh/?q=%25.target-corp.com&output=json
[*] 42 hosts added to the database.

Step 2: Resolve Discovered Subdomains to IP Addresses

1
2
3
4
5
6
7
# Load the resolver module which reads un-resolved hosts from the hosts table
[recon-ng][TargetCorp_Investigation] > modules load recon/hosts-hosts/resolve
[recon-ng][TargetCorp_Investigation][resolve] > run
[*] Resolving hostnames...
[*] api.target-corp.com resolved to 198.51.100.25
[*] vpn.target-corp.com resolved to 198.51.100.44
[*] 38 host records updated.

Step 3: Harvest Corporate Employee Contacts

1
2
3
4
[recon-ng][TargetCorp_Investigation] > modules load recon/domains-contacts/hunter_io
[recon-ng][TargetCorp_Investigation][hunter_io] > run
[*] Harvesting contacts for domain target-corp.com...
[*] 18 contacts added to the database.

4. Automated Report Generation

Once reconnaissance modules complete, compile all database tables into an executive dossier:

1
2
3
4
5
6
7
[recon-ng][TargetCorp_Investigation] > modules load reporting/html
[recon-ng][TargetCorp_Investigation][html] > options set CREATOR Abdulrahman M. Fathi
[recon-ng][TargetCorp_Investigation][html] > options set CUSTOMER Target Corp Executive Board
[recon-ng][TargetCorp_Investigation][html] > options set FILENAME /home/analyst/Reports/TargetCorp_OSINT.html
[recon-ng][TargetCorp_Investigation][html] > run
[*] Generating report...
[*] Report generated: /home/analyst/Reports/TargetCorp_OSINT.html
  • Opening this HTML file presents interactive, sortable tables detailing every host, IP, contact, and vulnerability discovered during the engagement.

Real-World Case Scenario

Case Study: Red Team Pre-Attack Surface Mapping in Under 30 Minutes

An offensive security firm was contracted to conduct a blind red team penetration test against a regional hospital system with over 200 physical clinics.

The lead intelligence analyst deployed Recon-ng:

  1. Ingested the hospital’s primary domain into a fresh workspace.
  2. Chained certificate_transparency and brute_hosts to discover 145 unique subdomains.
  3. Ran hosts-hosts/resolve to map all subdomains to IP blocks.
  4. Ran hosts-hosts/shodan_hostname to query exposed services across all resolved IPs out-of-band.
  5. Within 25 minutes, Recon-ng’s database flagged an unauthenticated DICOM medical imaging PACS server (pacs-archive.hospital.org) exposed directly on port 104 with public directory browsing enabled.
  6. The finding was documented in an automated HTML report and escalated immediately to hospital leadership for remediation before adversarial compromise.

Defensive Countermeasures & OpSec Recommendations

For corporate enterprise defenders and SOC managers:

  1. Conduct Recon-ng Self-Audits: Periodically execute Recon-ng sweeps against your own corporate domain names to discover what external threat actors can see in under 15 minutes.
  2. Scrub Public Contact Harvesters: Submit removal requests to data scrapers like Hunter.io and RocketReach to suppress corporate email naming conventions.
  3. Blacklist Scanning Engines at the Edge: Block known Shodan, Censys, and Project Sonar scanner IP ranges at the border firewall to prevent automated passive banner aggregation.

Conclusion & Next Steps

Chapter 22 proves that automation and relational database management transform scattered OSINT findings into cohesive, actionable intelligence. By standardizing module execution and chaining data flows, an investigator operates with maximum speed and consistency.

In Chapter 23: Radio Frequency & SIGINT, we expand beyond internet networks into open-source Signals Intelligence: Software Defined Radios (RTL-SDR), ADS-B aircraft flight decoding, and maritime AIS vessel tracking.


Practical Recon-ng Modular Intelligence Automation Cheat-Sheet

Recon-ng Command Operational Syntax Intelligence Function
Initialize Case Workspace workspaces create Case_TargetInvestigation Builds an isolated relational SQLite database container for current engagement
Add Target Seeds db insert domains targetcorp.com Injects initial domain or entity seed values into the reconnaissance database
Execute Modular Worker modules load recon/domains-hosts/certificate_transparency && run Harvests subdomains via SSL transparency logs and automatically stores them in db
Enrich Discovered Hosts modules load recon/hosts-hosts/resolve && run Resolves discovered subdomains to active IP addresses using multi-threaded DNS
Case Intelligence Summary dashboard && show hosts Outputs interactive visual metrics and structured table of all discovered assets

Authoritative References & Verified Investigative Sources


Author: Abdulrahman M. Fathi (0x3bdo) • Cyber Security Engineer & Penetration Tester

Open Research & Community Labs
صدقة جارية • Ongoing Charity

Empowering the Cybersecurity Community

All research, lab walkthroughs, and technical series on this platform remain 100% free. You can support server hosting, research environments, and lab infrastructure.

Support & Wall of Honor
اللهم اجعل هذا العمل علمًا نافعًا وصدقةً جارية، واكتب لي به الأجر والقبول، وانفع به كل من قرأه أو تعلّمه.
اللهم احفظ لي والديّ وأهلي، وبارك في أعمارهم وصحتهم، وأدم عليهم نعمة الصحة والعافية، ولا تريني فيهم بأسًا يبكيني.
Dedicated as ongoing charity and beneficial knowledge. May Allah bless my parents and family, protect them, and grant them lasting health and happiness.