A distributed, high-availability virtualization and autonomous AI laboratory engineered to enterprise reliability standards. Combining bare-metal hypervisors, distributed event queuing, zero-trust edge proxies, multi-terabyte storage pooling, and closed-loop automated self-healing.
Physical Compute Nodes
Managed Microservices
Hybrid Storage Fabric
Automated Self-Healing
All configurations, Docker Compose stacks, Traefik dynamic routes, and Ansible automation playbooks are version-controlled in Git. The infrastructure is tracked by a master executable lab specification and an indexed knowledge graph, eliminating configuration drift across all physical and virtual nodes.
A centralized perimeter reverse proxy powered by Traefik v3 and Authelia enforces Forward Authentication, OIDC (OpenID Connect), and Multi-Factor Authentication (TOTP / WebAuthn). Cloudflare DNS challenges manage automated wildcard SSL certificates, ensuring no internal administrative dashboard is directly exposed.
Distributed workflow execution across multiple compute nodes via a RabbitMQ message bus. An asynchronous n8n cluster leverages dedicated, isolated LXC sandboxes to safely execute AI-generated Python and shell code. Multi-agent DAGs run on LangGraph with LangSmith tracing, complemented by local Ollama/DeepSeek inference.
Full telemetry ingestion with Prometheus, cAdvisor, and Loki. When a service degradation is detected by 24/7 edge probing, an automated remediation pipeline pulls the trailing log buffer, performs LLM root-cause triage, and triggers target Ansible playbooks to self-heal without human intervention.
Four dedicated bare-metal computing nodes operating in synchronized orchestration across physical compute, NAS storage, and edge resilience.
Proxmox VE 8.x • Core Host
Proxmox VE 8.x • AI Cluster
Debian • Edge Diagnostics
Debian • Mesh VPN & UX
While Proxmox VE governs bare-metal hypervisors and Portainer handles raw container sockets, Homelab Central was custom-engineered from the ground up as a bespoke Next.js 16 & React 19 command center. It unifies operations across all 4 compute nodes, 20+ microservices, autonomous AI agents, and custom IoT hardware into a single intuitive control interface.
Dynamic inventory indexing 30+ services across Docker, LXC, and VMs. Categorized by tier (Infrastructure, AI, Security, Storage, Telemetry) with real-time health pings and single-click authenticated deep-links via Traefik & Authelia.
Direct telemetry ingestion from custom-designed ESP32 microcontroller nodes. Visualizes soil moisture capacitive curves, ambient temperature, humidity, and triggers automated solenoid irrigation valves with safety shut-off limits.
Mission control for the autonomous Hermes agent VM. Tracks active web intelligence scraping jobs, recurring agent tasks, LLM execution pipelines, and displays real-time execution logs and Telegram dispatch notifications.
Real-time observability of token velocity, context saturation, and API credit ceilings. Monitors local Ollama inference clusters (Llama 3, DeepSeek) and cloud routing gateways (OpenRouter) to prevent quota starvation.
Aggregates Prometheus time-series metrics, cAdvisor container load, and node-exporter telemetry across all 4 bare-metal hosts. Displays live CPU load, DDR4 saturation, NVMe flash IOPS, and network throughput graphs.
Direct execution tracking for n8n workflow queues and RabbitMQ event pipelines. Features manual dispatch triggers for Ansible self-healing playbooks, maintenance wave reboots, and storage scrubbing jobs.
An end-to-end interactive blueprint detailing how traffic flows from edge ingress, through the Homelab Central command plane, across bare-metal compute hosts, isolated Virtual Machines (VMs), lightweight LXC containers, and Docker microservices.
Core Automation & Local AI Docker Host
Hermes Autonomous AI Agent Platform
Home Assistant Supervised OS Core
5.4TB Unified MergerFS Storage Pool
Compute & Interface Docker Workloads
Dedicated, low-power edge guardian running 24/7 network inspection, primary recursive DNS, and health probing.
Host for the primary Homelab Central control plane application, WireGuard mesh routing, and telemetry dashboards.
Client requests hit Traefik v3 over TLS 1.3. Traefik triggers a Forward-Auth query to Authelia in edge-router-lxc. Once 2FA is validated, requests proxy to target services across Node 1, 2, or 4 over internal isolated virtual bridge networks.
Homelab Central (hosted on Node 4) queries the Proxmox VE cluster API, Prometheus TSDB, and Docker sockets via Portainer agents. Aggregates live system health, manages IoT plant sensor queues, and controls AI agent task schedules.
Hermes agent or n8n dispatches tasks to RabbitMQ. Code execution workloads run in the privileged n8n-sandbox-lxc container, while inference requests stream to local Ollama (Llama 3/DeepSeek) on hot NVMe storage.
Uptime Kuma or Alertmanager flags non-200 responses. Webhook dispatches to n8n Lab Medic, which pulls Loki logs, runs LLM root-cause triage, and triggers target Ansible playbooks to recycle containers with MTTR < 45s.
Segmentation between isolated Virtual Machines (VMs) for heavy workloads and unprivileged LXC containers for zero-overhead performance.
Primary Docker host running n8n Core orchestrator, RabbitMQ enterprise message broker, AnythingLLM, Promtail, and cAdvisor metrics.
OpenMediaVault 8 with hardware SATA controller passthrough (4TB Seagate IronWolf + 2TB WD Red). Pooled via MergerFS into a unified 5.4TB net share.
The security perimeter. Traefik v3 reverse proxy with Let's Encrypt wildcard SSL automation and Authelia Forward-Auth Single Sign-On.
Prometheus 2.x time-series engine, Grafana 10.x visualization dashboards, and Grafana Loki log ingestion with a 28-day retention window.
Privileged, dedicated container sandbox runtime. Allows AI agents to generate, test, and execute arbitrary Python and shell scripts without touching host filesystems.
Stateful Python multi-agent orchestration engine executing multi-step reasoning DAGs with LangSmith instrumentation for step-by-step token and latency auditing.
Separating ultra-low latency flash compute from resilient, expandable bulk storage.
High-throughput solid-state flash backing all virtual machine root disks, PostgreSQL relational databases, Redis memory stores, and active Docker volumes. Ensures instantaneous container startup and database query execution.
Rather than using fragile hardware RAID or rigid ZFS that mandates identical drive sizes, MergerFS combines a 4TB Seagate IronWolf and a 2TB WD Red into a single unified mount point. If any single drive ever fails, files on remaining drives stay 100% readable.
Observe → Alert → Analyze → Remediate. An automated event loop powered by LLMs and Ansible that diagnoses and fixes incidents autonomously.
24/7 edge health probes via Uptime Kuma and Prometheus Alertmanager detect service degradation or non-200 HTTP responses.
Alert triggers a webhook to the n8n "Lab Medic" orchestrator, which immediately pulls the last 50 log lines from Grafana Loki.
A local LLM evaluates the error buffer to distinguish between transient network timeouts and fatal application crashes.
If resolvable, n8n invokes Ansible's self_heal playbook to recycle target containers; otherwise, escalates to Telegram and Gotify.
System reboots and kernel upgrades are orchestrated via Ansible in a strict two-wave dependency chain. Wave 1 cycles stateless workers, proxy caches, and edge nodes. Wave 2 safely coordinates database checkpoints, storage unmounts, and hypervisor maintenance, eliminating deadlock states.
├── ansible/ │ ├── inventory.ini # Target node registry │ ├── playbooks/ # deploy.yml, self_heal.yml, reboot.yml │ └── vault.yml # Encrypted credentials & secrets ├── docker-compose/ # Master declarative service manifests ├── terraform/ # Proxmox VM & LXC provisioning specs └── LAB_SPEC.md # Master executable lab specification
The homelab operates as an active production laboratory. Beyond deployed infrastructure, here is the active engineering backlog and high-priority initiatives currently undergoing development, testing, and deployment.
Orchestrating an autonomous 4-bot agent swarm consisting of specialized personas (Manager, Monitor, Reviewer, and Executor) running cyclic LangGraph state machines in Python. Integrating FastMCP (Model Context Protocol) servers to provide sandboxed local LLMs with secure schema-validated tool-calling directly into homelab diagnostics, metrics, and Docker APIs over Telegram and Discord.
Eliminating the single point of failure in network name resolution. Currently, edge DNS runs on a primary node; implementing a mirrored secondary AdGuard Home recursive DNS resolver onto a secondary ARM64 node. Incorporating adguardhome-sync for bi-directional blocklist replication and configuring dual-resolver DHCP dispatch so client machines fail over seamlessly with zero downtime during maintenance reboots.
Engineering a stealth Playwright browser automation tool and reverse-engineered API scraping client integrated directly into the Hermes autonomous agent VM. Designed to periodically scrape and cross-compare multi-vendor product catalogs and rapid-commerce pricing data, identifying pricing discrepancies and pushing instant event alerts via RabbitMQ and Telegram.
Advancing from declarative service configuration to full lifecycle infrastructure automation. Finalizing one-click Proxmox LXC/VM provisioning via Ansible and Terraform providers, integrating strict Git pre-commit hooks to lint Docker Compose and Ansible YAML syntax prior to commit, and auto-generating human-readable architecture documentation directly from machine-readable JSON topology specifications.
Visual agent canvas (Langflow, Sim Studio) • Code structure indexing (CodeGraph AST) • High-throughput MCP agents (mcp-agent)