Sharing takeaways from attending Nvidia CEO Jensen Huang’s keynote yesterday at Nvidia GTC:
1. Generative AI continues to drive the need for ever bigger computing scale and acceleration. In turn, those new computing advancements enable significant improvements in Generative AI, i.e., it’s a symbiotic loop.
2. Nvidia’s new Blackwell architecture enables bigger AI models (up to 10 trillion parameters) with much faster throughput (claimed “up to a 30x” performance, e.g., tokens per second) and “up to 25x” reduction in energy consumption. This can support more capable models (e.g., multimodal) with a faster user experience.
3. While many still think of Nvidia as a semiconductor company, it continues to expand as a broader platform and infrastructure systems vendor. The size and complexity of those systems has increased significantly. The latest system weighs 3,000 lbs.
4. The innovations in these systems, particularly in the new Blackwell platform, go well beyond the traditional semiconductor innovations. For instance, Nvidia has made major strides in networking with their NVLink Switch (1.8TB/s), which is important for overall AI infrastructure performance.
5. Jensen repeatedly said that Nvidia is working on “the full stack.” Software investments are a big part. Besides embedding software in its systems, Nvidia is broadening its software platform. For instance, Nvidia launched NIM - Nvidia inference microservices built on its CUDA platform - to make deploying pre-trained, packaged generative AI models in applications and co-pilots easier and faster.
BTW - Nvidia’s software/services revenue crossed the $1B annualized run rate last year (though still a small % of the total). Today, Nvidia mostly monetizes S/W indirectly through differentiation, ecosystem building, demand for its systems, and customer/developer retention for the platform.
6. These new computing advancements, system-wide innovations, and platform & ecosystem developments are a key part of Nvidia’s strategy to defend its moat vs. competitors and new entrants, protect its lucrative share of the industry profit pool, and position the company at the core of GenAI.
7. Jensen also highlighted opportunities in robotics (announced Project GR00T, a foundation model for humanoid robots), digital twins, and “omniverse” (announced Omniverse Cloud APIs for industrial digital twin software tools), particularly for industrial, manufacturing, and logistics. AI is a key enabler for these, too.
8. While the keynote covered a lot of ground, one notable topic it didn’t discuss was responsible & ethical AI and AI security & governance, beyond Jensen briefly mentioning that the Blackwell platform has improved encryption to protect AI models and user data.
Overall, as “a developer conference,” Nvidia GTC has significantly broadened its audience, and the keynote at the SAP Center was packed. Some people compared it to a rock concert (and Jensen wore a new leather jacket).