HomeBlogBlog Detail

Introducing Anyscale KubeRay Connect: Doubling down on Kubernetes

Anyscale KubeRay Connect is now available in two tiers. 

- Launch tier is self-serve and starts with $500 in credits as part of self-serve free trials for new accounts. 

- Scale tier, designed for teams with multicluster and enterprise deployments, is now in early access. Request access.

If you run Ray on Kubernetes, you have probably already made your choices: KubeRay for the operator, Kueue or KAI for queueing, Argo CD for delivery, Kyverno for policy checks. That stack works, your team knows it, and your platform is built around it.

Until now, adopting the Anyscale Platform meant leaving those choices behind. The Anyscale Operator creates and manages Pods directly and does not understand KubeRay CRDs, so moving onto the platform meant rewriting your job specs, your GitOps repos, and the tooling wrapped around them.

Today we are introducing Anyscale KubeRay Connect, now available in two tiers, Launch and Scale. This new deployment format for teams running Ray on Kubernetes removes that tradeoff: you keep the KubeRay stack you already run, and get Anyscale's observability and orchestration on top of it. Your KubeRay Operator keeps reconciling your CRs exactly as it does today, and you keep submitting workloads the way you already do.

LinkWhy Anyscale KubeRay Connect

The gap between Anyscale and the KubeRay ecosystem cuts in both directions. KubeRay Connect closes it from both.

LinkOnboarding without migration

No migration, no CR rewrites. Previously, with no CRD support on the platform, your RayJob, RayService, and RayCluster specs had no path onto Anyscale, teams had to translate everything into Anyscale YAML first.

Now an engineer can point the Anyscale Connector at a KubeRay cluster that is already running, and Anyscale layers observability, scheduling, and lifecycle management directly onto the CRs already there. Setup is a Helm chart considerably lighter than the Anyscale Operator.

LinkInteroperability with the Kubernetes ecosystem

Rather than driving Pods directly, KubeRay Connect builds on KubeRay CRDs and leaves state reconciliation to the KubeRay Operator, which follows the standard Kubernetes controller architecture.

That distinction matters because of the large body of tooling built on top of Kubernetes native primitives like CustomResourceDefinitions (CRDs) and custom controllers. Kueue integrates with Deployments and KubeRay CRDs; Argo CD and Kyverno work the same way. Under KubeRay Connect, all of it keeps working.

LinkWhat it is

Instead of creating Pods, Anyscale mutates your existing KubeRay specs at admission, injecting lightweight dataplane sidecars that power observability. KubeRay Connect comes in two tiers. Launch is the self-serve entry point and the lowest-friction way to start, and Scale adds orchestration across your multi-cluster or multi-cloud compute estate.

Both tiers install the same egress-only connector and share the same observability, developer experience, and governance features. Scale adds scheduling to drive compute efficiency across clusters and clouds, runs Anyscale Jobs and Services in full instead of for observability only, and moves support from Developer to Enterprise. Launch is self-serve and can be tested with $500 in credits, Scale is purchased through our sales team. 

Launch

Scale

Best for

Teams early in their KubeRay buildout looking to quickly run root cause analysis and speed up debugging with full-stack observability. 

Teams seeking more efficient and reliable multi-cluster or multi-cloud compute through orchestration and dedicated support.

How to start

Self-serve free trial

Talk to us for early access

Credits

$500

Varies

Support

Developer

Enterprise

For more details on what is available, see our documentation.

LinkArchitecture at a glance

Anyscale KubeRay Connect installs two Anyscale components into your dataplane.

The Anyscale Connector is responsible for control operations: syncing state between the Anyscale Platform and your cluster, and serving a MutatingWebhook that patches KubeRay CRs at admission.

It is a Helm-installed Deployment, running on a single pod as of now.

The Cluster Telemetry Gateway is responsible for aggregating logs and metrics from your Ray workloads and ships them to the Anyscale control plane. It is an OpenTelemetry Collector, deployed with a small set of sidecars.

Architecture at a glance
Architecture at a glance

Splitting control from telemetry is a deliberate choice. The first benefit is fault tolerance – the Connector is only needed at the beginning of a pod's lifecycle, to mutate the spec; after that it can crash without affecting any metrics already flowing. The reverse holds too — a telemetry failure has no effect on admitting new workloads. This separation also yields both security (a split in identity that prevents bad actors) and scalability benefits (concurrent operations trigger Kubernetes autoscaling independently).

Three properties come out of this design:

  • Egress-only. Every connection is initiated from your dataplane. The Anyscale control plane needs no ingress into your cluster, ever.

  • Kubernetes-native authentication. The Connector authenticates with a projected service account OIDC token, verified against your cluster's own JWKS, removing the need for an Anyscale API key in the loop. This works on any conformant cluster whose JWKS we can fetch — it doesn't have to come from the issuer itself, and can be served from anywhere, even a bucket.

  • Resilient to control plane outages. Ray workloads keep running and keep making progress even if the Anyscale control plane is unavailable.

The mutating webhook is fail-closed by default. If the Connector is unavailable when a KubeRay CR is submitted, the CR is rejected rather than admitted without telemetry wired up. We chose this over fail-open to avoid silently admitting workloads in an incomplete state. The behavior is configurable.

If the Connector fails after a workload is admitted, telemetry keeps flowing, the Gateway is independent of it. What you lose is the job state: the Connector is what reports phase transitions back to the platform, so a job moving from pending to running to terminated won't update in the console until the Connector recovers. The workload itself is unaffected.

LinkWhat you get on day one

LinkObservability

Once the telemetry gateway is flowing, you get the Anyscale observability out of the box for your KubeRay workloads, and it works whether you submit through kubectl apply or through the Anyscale API:

  • Ray workload dashboards including Ray Data, Ray Train, and Cluster views, built for Ray 

  • Task and actor dashboards, which persist after the cluster terminates (Ray 2.50.0+)

  • Aggregated logs via Loki, plus per-workload logs

  • Workload, service, and job events, including autoscaler logs

  • Telescope, with a new full-stack view spanning your hardware to your Ray applications, across your fleet

That last one is worth highlighting: Telescope now correlates signals from the hardware layer (DCGM) through Kubernetes infrastructure events all the way up to your Ray applications — so a slow training job and the failing GPU underneath it show up in the same place. We wrote about that work in more detail in our recent full-stack observability post. 

This is the observability tooling Anyscale has built into the platform, now available on KubeRay workloads in minutes.

LinkOrchestration

With Scale, KubeRay Connect also covers workload submission, scheduling, and status tracking on Kubernetes:

  • Submit KubeRay CRs directly to the cluster with kubectl apply -f job.yaml, or through the platform with anyscale job submit -f job.yaml

  • Track job progress, status, and retries from the Anyscale console

  • Queue with Kueue, by applying Kueue configuration to your CRs

  • Queue with the Anyscale Scheduler, in Kueue passthrough or layered mode — coming soon

LinkWhat's next

The Launch tier covers onboarding and full-stack observability, and Scale adds single-cluster orchestration today. From here we are working on:

  • Multi-cluster orchestration. Available to Scale customers in early access, using health and availability signals from full-stack observability to place workloads across clusters.

  • Identity passthrough. Mapping cluster identity to platform identity so RBAC, attribution, and quotas work on the cluster submission (kubectl apply -f job.yaml) path.

  • For services, multi-region/cloud serving. Deploy a single Ray Serve application as multiple independent clusters across cloud providers and regions, managed as one service, so customers get scarce accelerator capacity, higher availability, and cross-provider disaster recovery.

LinkGet started

Launch is self-serve, and new accounts start with $500 in credits. Connect a Kubernetes cluster that already runs the KubeRay operator, and your Ray workloads show up in the Anyscale console with full-stack observability. You can let your coding agent run the setup for you, or run it yourself with the Anyscale CLI. Either way, setup checks your cluster first and stops before creating anything if a requirement is missing.

KubeRay Connect set up in less than five minutesKubeRay Connect set up in less than five minutes.
KubeRay Connect set up in less than five minutes

LinkBefore you start

  • Kubernetes 1.30 or later, with KubeRay 1.3 or later installed

  • Ray 2.50.0 or later in your RayCluster, RayJob, and RayService images

  • Anyscale CLI 0.26.108 or later

  • A DCGM exporter in the cluster if you want GPU metrics

  • Cluster-admin access to install the connector

LinkRun setup yourself with the CLI

The CLI walks you through a guided setup that takes about three minutes end to end. Install it, log in (a browser window opens so you can approve access to your organization), and point anyscale cloud setup at your cluster:

pip install -U anyscale
anyscale login
anyscale cloud setup --stack kuberay \
  --name my-kuberay-cloud \
  --namespace my-namespace \
  --cluster-context arn:aws:eks:us-east-2:<account-id>:cluster/<cluster-name>

It inspects the cluster, reuses what’s already there (KubeRay operator, DCGM exporter, ingress controller), shows you a plan, and asks before changing anything:

Setting up KubeRay cloud 'my-kuberay-cloud'
Checking for a KubeRay operator on this cluster...
Checking for a DCGM exporter on this cluster...
Checking for ingress controllers on this cluster...
  Provider:           AWS (us-east-2), inferred from the token issuer
  Namespace:          my-namespace
  Storage & IAM:      CloudFormation stack 'anyscale-kuberay-my-kuberay-cloud' (S3 bucket + IAM role)
  KubeRay operator:   found: kuberay-operator (quay.io/kuberay/operator:v1.6.2)
  GPU metrics:        found: monitoring/dcgm-exporter
  Ingress:            install the traefik ingress controller bundled with the connector chart
Continue? [y/N]: y

It then provisions storage and IAM, registers the cloud, installs the Connector, and waits for it to come up healthy:

CloudFormation stack anyscale-kuberay-my-kuberay-cloud completed successfully
Cloud registered with ID: cld_<your-cloud-id>
Connector installed
Connector is HEALTHY

KubeRay cloud setup complete!
  Cloud ID:          cld_<your-cloud-id>
  Cloud resource ID: cldrsrc_<your-cloud-resource-id>
  Object storage:    s3://anyscale-kuberay-my-kuberay-cloud-<suffix>
  Run RayJobs in:    my-namespace

Run a sample RayJob now to see logs and metrics in Anyscale? [Y/n]:

When it finishes, it offers to submit a sample RayJob so you can see logs and metrics in the console right away.

Running GitOps?

The connector also ships as the anyscale-connector Helm chart. Register the cloud with the CLI, put the IDs in your values file, and let Argo CD install the chart like the rest of your stack:

anyscale cloud register -f kuberay-cloud.yaml --provider generic --name my-kuberay-cloud

yaml

# values.yaml
cloudId: cld_<your-cloud-id>
cloudResourceId: cldrsrc_<your-cloud-resource-id>
helm install anyscale-connector anyscale/anyscale-connector \
  --namespace anyscale --create-namespace -f values.yaml

Submit a workload the way you already do

kubectl apply -f rayjob.yaml

The job appears in the Anyscale console with its Ray dashboards, logs, and events. Nothing about how you submit changes. Full setup guide: docs.anyscale.com/kuberay-connect/setup.

Get started with Launch tier with $100 in credits as part of a free trial for new accounts. 

Explore Anyscale today

Build, run, and scale any AI workload on Ray with a multi-cloud platform built for production AI.