Ray manages, executes, and optimizes compute needs across AI workloads. It unifies infrastructure by leveraging any compute instance and accelerator on AWS via a single, flexible framework—enabling any AI workload from data processing to model training to model serving and beyond.

Started as a research project in 2016, Ray today is used by leading AI organizations like Uber, Spotify, Netflix, Pinterest, OpenAI, and many more. Across the ecosystem, it is powering 1 million clusters each month for AI and GenAI workloads (and growing).

At Ray Summit 2024, Anyscale released an optimized version of Ray, called RayTurbo, offering improved performance, efficiency, scale, and reliability.

Join this session to learn more details about RayTurbo including how it delivers:

• Up to 4.5X faster data processing

• Up to 6X lower costs for LLM Batch Inference compared to Bedrock

• Up to 60% lower costs on many workloads by supporting spot instances with elastic training

• Up to 50% fewer nodes for online model serving with Replica Compaction for better efficiency.

• Up to 60% higher QPS serving with optimized version of Ray Serve

Speakers

• Richard Liaw, Product Manager Anyscale

• Edward Oakes, Staff Engineer Anyscale