Built a shared Kafka platform that IT and other application teams plug their own apps into, instead of each team running its own broker.
3-broker cluster in KRaft mode (no ZooKeeper), replication factor 3, every node running broker, controller, and JMX exporter
Separate tooling VM with AKHQ for topic and consumer lag inspection, Prometheus scraping broker JMX metrics, Grafana dashboards, and Redis for consumer services
nginx reverse proxy in front of all admin UIs
Specified broker sizing (4 vCPU, 8 GB RAM, 50 GB OS + 300 GB data, RHEL 8.10, non-root) and the full port and firewall plan
Built with