Dr. Ronen Dar (Co-Founder/ CTO of @runailabs) talks about the challenges of running compute infrastructure for AI, the GPU ecosystem, sizing LLMs and more.
SHOW: 739
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT - "CLOUDCAST BASICS"
SHOW SPONSORS:
- GCore - Global Hosting, CDN, Edge and Cloud Services
- Use promocode “CLOUDCAST” to receive a €100 credit on Gcore services
- Equinix Global Data Centers and Networking
- Learn more and signup at https://deploy.equinix.com/. Use the coupon code CLOUDCAST to get $500 in credits to get started.
- Datadog Kubernetes Solution: Maximum Visibility into Container Environments
- Start monitoring the health and performance of your container environment with a free 14 day Datadog trial. Listeners of The Cloudcast will also receive a free Datadog T-shirt.
SHOW NOTES:
- Run:ai - Build Your Next Large Model (homepage)
- Ronen Dar, Run:AI's CTO, on managing computation resources in ML pipelines
Topic 1 - Welcome to the show. Tell us a little bit about your background, and what you focus on at Run:ai.
Topic 2 - Let’s begin by talking about the challenges of running AI applications. What unique characteristics and requirements do AI applications have?
Topic 3 - Most AI applications run on GPUs. How do things change when using GPUs vs. CPUs to power AI applications? What is needed to get the most out of GPUs?
Topic 4 - As environments grow larger, what is needed to scale-up environments, both in terms of scheduling applications and managing the underlying GPU infrastructure?
Topic 5 - GPUs are not only expensive resources, but also in high-demand. How are companies doing capacity planning with GPUs? What struggles are you seeing companies have as they manage planning for AI projects?
Topic 6 - Are the new Large Language Models (LLMs) much different in size than AI models of the past?
Topic 7 - How well is the industry prepared to deal with the new interest in AI from across the industry?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @thecloudcastnet
