From Infrastructure to Production: A Year of Self-Hosted LLMs

Abstract

Last year, I published Paddler, an open-source LLM load balancer. A year of community feedback and building Poet (a static site generator with AI features) on top of it taught me what actually matters when self-hosting LLMs. This talk shares practical patterns the open-source community needs. What works, what doesn’t, and what tooling we still need to build together as a community.

Bio

Mateusz Charytoniuk is a Technical Founder at Intentee, where he leads the development of innovative conversational applications builder. Mateusz is also involved in projects like Paddler, a llama.cpp-based load balancer. His expertise is further showcased through his active Open Source contributions.