Welcome Keerti
Today, I’m thrilled to welcome Keerti Melkote as our new CEO. A visionary who founded Aruba Networks in his (literal) garage in 2001, Keerti led the company through monumental growth, taking it public in 2007 and reaching over $5 billion in revenue. Very few founders have taken a startup all the way through every scale of growth the way Keerti has, and I'm overjoyed to be working with him as we take on the massive AI infrastructure buildout over the coming decade.
When we started Ray and Anyscale, we had a simple story in mind and a simple bet:
AI will transform every business and every industry.
AI workloads are becoming more and more compute intensive.
The scale required by AI workloads is creating hard infrastructure challenges.
This bet on scale is now playing out.
That said, for the first few years of the company, the bet was not obvious. The number one question people would always ask us was whether distributed computing was really necessary. Isn’t a single machine enough? It wasn’t until ChatGPT and the generative AI wave that the need for scale became abundantly clear.
Since then, Ray has taken off. To name a few examples:
Pinterest runs 5000 training jobs and 300 batch inference jobs every month on Ray.
Amazon processes exabytes of data with Ray.
Runway used Ray to help build Gen-3 Alpha.
OpenAI used Ray to train GPT-4.
Uber standardized model training on Ray.
Ant Group serves over one million requests per second with Ray.
Similar stories abound at Canva, ByteDance, Instacart, Spotify, Niantic, Samsara, Netflix, Airbnb, Adobe, Shopify, Zoox, LinkedIn, and Handshake.
Many more will be shared at Ray Summit 2024 in October.
Our business has transformed over the past year. We’re on track to quadruple revenue this year, and Ray adoption is growing 6x year over year. Our customers regularly run workloads on hundreds of thousands of CPU cores and many thousands of GPUs, spinning up 5000-node clusters in minutes.
Over the past year, we’ve enabled pioneering companies to push the frontiers of AI. The industry is barely scratching the surface of what’s coming, but many of the trends are very clear.
Data-intensive workloads: In addition to becoming more and more compute intensive, AI workloads are about to become far more data intensive. Today, many of us work with LLMs and text data. Soon we’ll treat video and audio and all sorts of multimodal data the way we treat text. Video data is vastly bigger, so AI applications will become far more data intensive.
AI in data preparation. Far more AI is being used in data preparation today. The quality of training (or fine-tuning) data plays such a big role in the quality of the resulting model that more and more AI is being used to curate and augment the training data. This includes using classifiers to filter out low quality data, running inference to generate synthetic data, using LLMs to rewrite and catch mistakes in documents, using visual models to generate image captions, deduplication, and so on. The amount of AI and compute being used for unstructured data preparation and processing is about to skyrocket.
Exploding inference complexity. Historically, AI inference has been about hosting a model behind an endpoint. Now, we’re beginning to see AI workloads that involve hundreds of model calls to perform a single action (completing a task like booking an Airbnb, sending an email, or writing some code) intermixed with custom application logic. As many model calls compose together to power advanced agentic experiences, the latency and cost issues compound. Inference workloads one year from now will have vastly different characteristics from typical inference workloads today.
AI is still in its infancy. Every team is in the process of becoming an AI team, from growth to security to HR. Businesses are only just starting to think about unifying and optimizing the platforms they provide for all of these teams to efficiently build with AI and move to production. This is an incredibly exciting time for Anyscale, and it’s why I’m so excited to welcome Keerti today. His deep experience navigating nascent markets and scaling hypergrowth businesses will be invaluable as we build out the infrastructure to power AI in the years ahead.
We have a tremendous amount of work ahead of us. If you’re interested in playing a role in building the foundational infrastructure powering AI, please reach out. We’re hiring across the board and would love to work together.
