BentoML

As the year wraps up, let's explore the 10 most popular projects from #BentoML and our amazing community partners: 🚀 One-command #LLM deployment as OpenAI-compatible APIs with #OpenLLM https://lnkd.in/grWgC5VW 🤖 Self-host open-source LLMs with BentoML and #vLLM as OpenAI-compatible APIs https://lnkd.in/gygrPu98 🎨 Serve diffusion models like SD 3.5, #SDXL, and #ControlNet with BentoML https://lnkd.in/gfVRSACF 🛠️ Deploy #ComfyUI workflows as scalable APIs with comfy-pack https://lnkd.in/g8isREbV 🔍 Build private #RAG systems with open-source models using BentoML and #LlamaIndex https://lnkd.in/g5DqnCi3 📚 Serve #ColPali with BentoML for efficient multi-vector embeddings https://lnkd.in/gpyZGvaj 📞 Build a phone-calling #AI agent using open-source models with BentoML https://lnkd.in/gafTj4m3 🤝 Build multi-agent systems with #CrewAI and BentoML https://lnkd.in/ggGE8AvN 🧠 Build a #LangGraph agent application with an open-source model https://lnkd.in/gUwYtjnm ⚙️ Add function calling capabilities to open-source LLMs https://lnkd.in/g3J7ZqVh More to explore 👉 https://lnkd.in/geHXP5GN

🍱 BentoML Newsletter: Win A $300 Visa Gift Card! Share Your Insights on AI Inference Infrastructure in Our 2-Minute Survey

🎁 Complete our survey to win a $300 Visa Gift Card! 👉 https://lnkd.in/gxVn3Dhd 🚀 We’re conducting a 2-minute survey on the Status of AI Inference Infrastructure and would love to hear from you! Share your experience with deployment patterns, infrastructure challenges, GPU usage, model adoption, and more! Respondents will get early access to exclusive insights and be entered into a raffle for a $300 Visa Gift Card! Thank you for your participation and for helping us understand the landscape of AI infrastructure! #MachineLearning #AI #OpenSource #AIInfrastructure #BentoML

🎉 Introducing Comfy-Pack: Package, share, and deploy any ComfyUI workflows with ease! Replicating a 𝗖𝗼𝗺𝗳𝘆𝗨𝗜 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄 from one environment to another can be a messy and frustrating process. Keeping track of model files, managing dependencies, and ensuring consistency across setups is often a challenge—making deployment unnecessarily painful. With 𝗖𝗼𝗺𝗳𝘆-𝗣𝗮𝗰𝗸, you can take any ComfyUI workflows from local prototypes to production-ready deployments in no time: ✅ 𝗣𝗮𝗰𝗸𝗮𝗴𝗲 workflows into reproducible "cpack" files ✅ 𝗗𝗲𝗳𝗶𝗻𝗲 𝗮𝗻𝗱 𝘃𝗮𝗹𝗶𝗱𝗮𝘁𝗲 𝗶𝗻𝗽𝘂𝘁𝘀 with standardized parameter nodes ✅ 𝗥𝘂𝗻 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀 directly via CLI ✅ 𝗔𝘂𝘁𝗼-𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲 𝗥𝗘𝗦𝗧 𝗔𝗣𝗜𝘀 with OpenAPI docs ✅ 𝗢𝗻𝗲-𝗰𝗹𝗶𝗰𝗸 𝗱𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁𝘀 to scalable infrastructure on BentoCloud 🛠️ Try it out: https://lnkd.in/ggwwWP2G 📺 Demo: https://lnkd.in/gCezu4qR #ComfyUI #ComfyPack #BentoML

🚀 Introducing comfy-pack – a comprehensive toolkit that converts ComfyUI workflows into robust, scalable APIs! ComfyUI has revolutionized AI art creation with its community-driven ecosystem, offering creators a wealth of resources like custom nodes for easy experimentation. But what happens when you want to deploy ComfyUI workflows at scale? ❌ No standard API interface ❌ Limited portability ❌ No scaling capabilities Here's where comfy-pack comes in. It helps you: ✅ Serve workflows as API endpoints with OpenAPI docs ✅ Package the entire workspace for reproducibility ✅ Deploy to BentoCloud for scalable, reliable cloud APIs Go from local prototyping to scalable cloud APIs in just a few clicks! Read our detailed blog post to learn more: https://lnkd.in/g8isREbV #AI #ComfyUI #BentoML #comfypack #MachineLearning #AIArt #ImageGeneration #OpenSource

Tired of the deployment headaches with #LLM agents? BentoML takes care of the heavy lifting - from REST endpoints to scaling. Great post showing how to combine #Burr and #BentoML for streamlined agent deployment. Read the blog post to learn more: https://lnkd.in/gJNccqw9

✨🚀 Have you tried Llama 3.3 yet? This new 70B model approaches the performance of Llama 3.1 405B, with simpler, more cost-efficient inference! 🦙💡 🔗 Serve Llama 3.3 with BentoML today: - fp16 precision: https://lnkd.in/g8WZxSiS - AWQ quantization: https://lnkd.in/gueRcVKC 💻 The code is ready to go, and you can easily extend it with additional functionalities, like function calling! #AI #Llama33 #BentoML #OpenSource #LLM

⚡️ BentoML Codespaces delivers 20x faster iteration! 🍱☁️ Connect your local environment to BentoCloud for powerful GPUs and view real-time updates as you make changes! Check out the demo on prompt updates for a voice agent with Codespaces! 👇 #BentoML #AI #BentoCloud #OpenSource #Codespaces

Developing #AI voice agents isn't just about selecting components — it's about overcoming complex challenges. Imagine juggling: - AI models (#STT, #LLM, #TTS) - Transport service - Pipeline management - Voice Activity Detection (VAD) The headaches? Lack of GPU access, slow iteration cycle, and inconsistent behaviors between dev and prod 😓. Watch this short video to learn how #BentoMLCodespaces solves these challenges. And stay tuned for our next video demo about hands-on development with Codespaces! #BentoML #BentoCloud #OpenSource #Codespaces

When building modern AI applications like voice agents, you have a choice between open-source models and managed APIs. Both options come with their unique advantages. Check out this video as our Head of Engineering Sean Sheng explores the benefits of open-source models, like: 🔒 Security and privacy 🛠️ Advanced customization 🧠 Predictable behaviors #AI #MachineLearning #OpenSource #BentoML