LinkIntroduction

Five years ago, if you asked a marketer how they created personalized marketing content, they would describe – essentially, a MadLibs game. “Personalized” content meant templatized emails, with pre-selected sections where you could plug in additional information depending on what you knew about your customers.

But true personalization is more than that – it’s content written from scratch that speaks to a customer’s exact needs, based on their shopping history and preferences. And it’s no longer science fiction—the Attentive AI team is making it a reality. 

To power bespoke and personalized shopping experiences for their customers, Attentive is investing heavily in AI and Machine Learning (ML). Innovating to create the future of shopping brings with it a host of infrastructure challenges and complexity. Attentive tackled these challenges with the same approach that goes into the design and innovation of its new products in lock step with their customers and a focus on delivering unmatched value.

LinkThe Scale Challenge

To learn more about Attentive’s AI journey, we met with Christian Stano, Engineering Manager for the ML Platform team at Attentive. He tells us about Attentive’s early investments in AI and ML infrastructure, and how they are tackling what he calls the “S curve” of complexity and scale, a key inflection point for ML platforms where complexity quickly increases, blocking the next level of performance and scale.

Fig 1. The "S curve" of ML complexity (Source)

“At the beginning of building an AI initiative, you see a bunch of early wins. But as you evolve your initiative, you start to run into the problem of scale. In order to build more complex models, you need to process and train on substantially more data.”

Scaling laws are not a new concept in AI: they were theorized as early as the 2000s, but it wasn’t until the late 2010s that model and dataset sizes really became a challenge. Teams like Christian’s are on the front lines of tackling, and overcoming, these challenges. They’re already investing in their next generation of ML infrastructure to future-proof their platform and be ready to tackle both current and future AI problems at scale. 

At the beginning of the “S curve," as Christian calls it, he and his team of three ML Ops engineers were spending their time managing, expanding, and monitoring their infrastructure setup. They used a variety of tools across the AI lifecycle, including Spark for data processing, PyTorch for model training, and Kubernetes containers to run their programs. With these tools, Christian and his team were constantly facing overhead and design tradeoffs in areas like cost management and self-service.

“We had tools in place. A home-grown Kubernetes solution that we self-managed and a lot of integrations we’d set up to work in the short term, but we knew it wasn’t going to work for the scale and vision we wanted to grow into.”

Attentive’s AI strategy is built on a foundation of scalable ML infrastructure, including a centralized feature store, robust model monitoring, and automated deployment pipelines. As Attentive expanded its AI capabilities, scaling compute became a bottleneck – leading the team to explore solutions like Anyscale.

LinkFinding a Solution, On a Deadline

Christian knew there had to be a better solution out there and he needed one he could implement quickly. “One of the biggest considerations for us was the implementation timeline,” he says. “I was leading a new team with lofty goals and a lot of product objectives to hit in a very short time.” 

Christian’s team quickly landed on Ray as their AI Compute Engine. As a Python-native framework, Christian knew it would be easier for the team to adopt and set up in the shortest amount of time possible. The clock, after all, was ticking. 

Initially, Christian’s team looked into KubeRay, since they already had a Kubernetes cluster management system set up. KubeRay is an open source Kubernetes operator that automates the deployment and scaling of Ray clusters. But though KubeRay would provide a shortcut to help Christian’s team get up and running, it was still a steep learning curve.

“We briefly considered going with KubeRay, but we needed to balance the scale and complexity of our project while minimizing dedicated engineering time on infrastructure. We are a small and scrappy team – after evaluating the complexity to set it up and the time it would take to reach our desired developer experience, Anyscale became the obvious choice. Anyscale accelerated those timelines for us so we could hit our goals.” 

LinkOnboarding Anyscale

Based on their mission to deliver personalized experiences, Christian and his team chose Anyscale as their AI Compute Platform. The Attentive team quickly dove into the Anyscale platform, exploring whether or not it would support Attentive’s high-volume data use cases.

“When we trialed Anyscale, we were amazed by how much just worked out of the box. The hosted workspace, the managed compute layer, autoscaling, spot instances… it would have taken us months to build what we got with Anyscale in 3 days.”

With Anyscale, the Attentive team could immediately: 

• Support coding, testing, and deployment in the same developer environment

• Integrate with their existing open source tools

• Monitor deployments and jobs in one place

• Select accelerators, manage clusters, configure autoscaling, and configure spot instance support in one place

• Control accelerator usage and spend with built-in tracking and alerts if things went out of budget

• Upskill their team, with one tool for data processing, model training, and model serving. 

The Attentive team decided to structure the Anyscale migration around models; migrating one model at a time and onboarding the machine learning engineers who work on that model to use Anyscale. As Attentive has migrated models, each model has reached new scales of performance and complexity, incorporating billions of data points. 

“Before Anyscale, we didn’t have the ability to consolidate our data into a single model because we simply couldn’t process it all. With Anyscale, we were able to unify it into one model and reduce the cost by 99% while increasing the data volume for it by 12X.”

Thanks to the partnership with the Anyscale team of experts, the team had their first two models onboarded onto Anyscale in just 45 days. In addition to getting access to the Anyscale platform, Anyscale engineers also went on site to train Attentive engineers on how to use Ray Train, Ray Tune, and Ray Serve.

“Because our team primarily uses open source tools, I know firsthand how difficult onboarding and ramp-up can be. Our experience with Anyscale was fantastic, and their support team has been a huge success factor for us. We even have an open Slack channel with them where we can ask questions and get the help we need immediately.”

LinkAttentive Today

By switching from MadLibs-style marketing personalization to truly personalized content powered by AI, Attentive empowers their customers to deliver true 1:1 marketing across multiple channels, improving sign ups, campaigns and journeys with the largest customers. With the help of their AI-powered products, Attentive drove more than $29 billion in revenue for our customers in 2024, an increase of 48% year over year. Attentive’s customers also see:

• 2x+ higher list growth

• 2x+ higher ROI

• 2.4x higher CVR with SMS + email

And they’re only getting started. Even though they’re only 50% of the way through their Anyscale adoption process, Christian’s team are already seeing amazing results, including:

• No ceiling on data, which means no ceiling on model training and quality

• No limit on compute options, giving Christian’s team full control over spend, speed, and efficiency

• Multimodal data processing out-of-the-box, making it easier than ever for the Attentive team to choose their compute, their way.

Developers across Attentive use Anyscale with a variety of flexibility, too, giving Christian the oversight that he needs without limiting them. Depending on how nuanced the developers want to get with their accelerator selection, they can either use Anyscale’s Smart Instance Manager to auto-select nodes, or they can specify the nodes they want to use directly within the platform. And with Anyscale’s advanced tagging and alerting capabilities, Christian and team get directly notified anytime spend goes out of bounds so they don’t have to constantly monitor it themselves. Right now, the team is small enough that they don’t need to establish compute and usage configurations by person, but that’ll always be in the team’s back pocket should they need it.

LinkAttentive Tomorrow

Onboarding Anyscale was just the beginning. The Attentive team has ambitious plans ahead for their continued migration and expansion with Anyscale. 

“We’re only about 50% of the way through the process of moving all of our models onto Anyscale. Looking into 2025, in addition to completing our model migration, we also plan to add new types and iterations of models using the scale Anyscale supports.”

Once all of their workloads are migrated, Christian and team plan to standardize operations on Anyscale and explore additional use cases within the Anyscale platform. 

One of the most exciting areas is around unifying and co-locating data processing. Working with the Anyscale team, Christian plans to increase the amount of data they process in their models and explore processing data on heterogeneous (CPU/GPU) clusters. Additionally, last mile data processing is still a manual process for the team right now, so he’ll also be exploring how to consolidate that in Anyscale. 

“We’ve built a strong relationship with Anyscale and we’re excited about the future. We’re in a very good position with the infrastructure that we have to support the next several months of use cases and the future looks very promising with the positive impact we can have on our customers.”

For more insight into how Attentive is driving the future of AI, check out their blog →