OSINT Research Environment

A self-contained, network-isolated research environment for open-source intelligence work. Built on a dedicated Proxmox LXC container with a non-root research user, curated toolset, and deliberate separation from production services.

Why Build This

OSINT work involves visiting unknown sites, running aggressive network scanners, and handling data from potentially hostile sources. Running this on a shared system risks contaminating production services if something goes wrong.

A dedicated isolated container solves this cleanly — if something goes sideways during an investigation, it stays contained. The container is intentionally stopped when not in use, reducing the attack surface further.

Toolset

Architecture

Proxmox LXC (Ubuntu 22.04)
    │
    ├─ root — system administration only
    │
    └─ osint user — all investigation work
           ├─ Python tools (~/.local/bin)
           ├─ Go tools (~/go/bin)
           └─ API key store (~/.osint-env, chmod 600)

All investigation work runs as a non-privileged osint user. API keys for paid services (HIBP, Shodan, VirusTotal) are stored in a permission-restricted environment file sourced at login.

Technical Highlights

• Network isolation — the container sits on its own network segment, preventing any lateral movement to production services
• Non-root operations — all tool installation and investigation work runs as a dedicated osint user, not root
• Go toolchain — the Project Discovery suite (subfinder, httpx, nuclei) requires a current Go installation; Ubuntu’s default repositories are too old and were replaced with a manual Go install
• Dependency management — recon-ng’s pip packaging is broken upstream; solved with a git clone and a bash wrapper script to maintain clean invocation
• Stopped by default — the container is excluded from the AI monitoring agent’s restart watch list and started only when needed

Challenges Overcome

• Node.js version conflicts — Ubuntu 22.04 ships with an outdated Node.js that breaks npm installs silently. Solved by removing system packages entirely and replacing with NodeSource v20.
• PATH complexity — Go binaries, pip user installs, and custom wrapper scripts each have different binary locations. Required careful .bashrc configuration to make all tools discoverable.
• Dependency sprawl — theHarvester has many optional async dependencies (aiodns, aiosqlite, playwright) that produce install warnings but don’t affect core functionality. Distinguishing fatal from non-fatal errors took time.

Tech Stack

• Ubuntu 22.04 LXC (Proxmox)
• Python 3 (pip user installs)
• Go 1.21 (Project Discovery toolchain)
• Node.js 20 (dashboard)
• Sherlock, Amass, Subfinder, httpx, Nuclei, theHarvester, recon-ng, Photon

Current Status

Built and verified — all 8 tools confirmed working. Kept stopped when not in use. Started on demand for investigations.

What I Learned

The hardest part of building an OSINT environment isn’t the tools — it’s the environment. Getting Go, Python, Node.js, and custom wrapper scripts to coexist cleanly on Ubuntu requires careful attention to PATH ordering and which user installs what. The isolation architecture also forced me to think more carefully about what “network isolation” actually means in a Proxmox environment and where the real boundaries are.

Part of an ongoing homelab security research project.