ComfyUI’s Post

An important update from PyTorch

The Attention Release!!! This one includes 2 major features and 1 hidden one that I think secretly could have the most impact: 1. CuDNN backend for SDPA is awesome, thanks Eddie Y. for all your hard work here and checkout this tweet thread for some comparisons: https://lnkd.in/gcUk5vua 2. FlexAttention in prototype, it has been really exciting to see the community already build on this in the nightlies, read the blog if you haven't: https://lnkd.in/gxg9BZRt 3. Making SDPA safe, nans be gone. While working on integrating FlexAttention w/ TorchTune the problem of fully masked out rows (masking out padding tokens) raised its ugly head again. We decided to change the behavior for this case. A longer description of the problem and solution can be found here: https://lnkd.in/gXuMA5fU

PyTorch 2.5 is here 🔥 We are excited to announce the release of PyTorch® 2.5 featuring: 🔥 a new CuDNN backend for SDPA, enabling speedups by default for users of SDPA on H100s or newer GPUs 🔥 regional compilation of torch.compile, offering a way to reduce the cold start up time for torch.compile by allowing users to compile a repeated nn.Module (e.g. a transformer layer in LLM) without recompilations 🔥 TorchInductor CPP backend offering solid performance speedup with numerous enhancements like FP16 support, CPP wrapper, AOT-Inductor mode, and max-autotune mode Read more in our PyTorch 2.5 Release Blog: https://hubs.la/Q02TRx6f0

Getting Started With GPIO Pins on Raspberry Pi (Beginner’s Guide) https://lnkd.in/da5MjcTQ

Cut LLM inference costs by mixing GPU types! From Tyler Griggs Introducing Mélange, a framework to derive minimal-cost GPU allocations for a given LLM service. X thread 👉 https://lnkd.in/djtpWswt The large and growing GPU market presents opportunities to exploit GPU heterogeneity and slash LLM costs. Mélange is easily extensible to new GPUs and LLM services, and we've made all of our work open source for others to try! Blog 👉 https://lnkd.in/dJad2PY6 Paper 👉 arxiv.org/abs/2404.14527 GitHub 👉 https://lnkd.in/dBjzq_sQ #machinelearning

A small piece of code that I wrote just to practice rust a bit more. What's more exciting though, is that it runs on a RISC-V CPU :) specifically the esp32c3 SoC. I had also created a demo project here: https://lnkd.in/efEEh66e https://lnkd.in/eYAMapHm

That’s a highly recommended read for requests and limits in k8s The kube scheduler and the Linux kernel are two separate beasts when it comes to resource management. It’s interesting to understand the somewhat static nature of how the kube scheduler decides where (on what nodes) to schedule pods. Follow the 4-part series to get a hands-on understanding of what kube does and how it translates into the lower level cgroup primitives. Hope you enjoy this read! #kubernetes #scheduling

In this 4-part series, you will explore how Kubernetes requests and limits work and how: - CPU requests and limits abstract over the Complete Fair Scheduler. - Memory maps to cgroups and OOM. - They affect node pressure and eviction More: https://ku.bz/MkwX-Wgcy

Got a Raspberry Pi? Here are 5 super handy commands every Pi user should know! From checking performance to managing GPIO pins, these tips will make your life easier. Swipe through to learn more! #raspberrypi

Banana Pi BPI-AIM7 with Rockchip RK3588, 6 TOPS computing power NPU, and can be equipped with up to 32GB of large memory. While the interface is fully compatible with JetsonNano https://lnkd.in/gazyVwRw