Introducing Anyscale KubeRay Connect: Doubling down on Kubernetes
If you run Ray on Kubernetes, you have probably already made your choices: KubeRay for the operator, Kueue or KAI for queueing, Argo CD for delivery, Kyverno for policy checks. That stack works, your team knows it, and your platform is built around it.
Until now, adopting the Anyscale Platform meant leaving those choices behind. The Anyscale Operator creates and manages Pods directly and does not understand KubeRay CRDs, so moving onto the platform meant rewriting your job specs, your GitOps repos, and the tooling wrapped around them.
Today we are introducing Anyscale KubeRay Connect, now in private preview. It removes that tradeoff: keep the KubeRay stack you already run, and get Anyscale's observability and orchestration on top of it. Your KubeRay Operator keeps reconciling your CRs exactly as it does today, and you keep submitting workloads the way you already do.
LinkWhy Anyscale KubeRay Connect
The gap between Anyscale and the KubeRay ecosystem cuts in both directions. KubeRay Connect closes it from both.
LinkOnboarding without migration
No migration, no CR rewrites. Previously, with no CRD support on the platform, your RayJob, RayService, and RayCluster specs had no path onto Anyscale, teams had to translate everything into Anyscale YAML first.
Now an engineer can point the Anyscale Connector at a KubeRay cluster that is already running, and Anyscale layers observability, scheduling, and lifecycle management directly onto the CRs already there. Setup is a Helm chart considerably lighter than the Anyscale Operator.
LinkInteroperability with the Kubernetes ecosystem
Rather than driving Pods directly, KubeRay Connect builds on KubeRay CRDs and leaves state reconciliation to the KubeRay Operator, which follows the standard Kubernetes controller architecture.
That distinction matters because of the large body of tooling built on top of Kubernetes native primitives like CustomResourceDefinitions (CRDs) and custom controllers. Kueue integrates with Deployments and KubeRay CRDs; Argo CD and Kyverno work the same way. Under KubeRay Connect, all of it keeps working.
LinkWhat it is
Instead of creating Pods, Anyscale mutates your existing KubeRay specs at admission, injecting lightweight dataplane sidecars that power observability. The product is structured in three tiers, and we are shipping them in order. Observe is the entry point, and deliberately the lowest-friction one.
Tier | What it does | Status |
|---|---|---|
Observe | Anyscale automatically collects full-stack telemetry, from the hardware up to your Ray workloads, and surfaces it in the Anyscale console: Ray dashboards, logs, events, and metrics. | Private preview |
Orchestrate | Anyscale’s intelligent scheduling layer places workloads across clusters using availability and health signals from Observe, or a scheduling strategy you define, to maximize utilization. | Private preview |
Optimize | Price-performance tuning of Ray and Ray dataplane knobs. | Coming soon |
LinkArchitecture at a glance
Anyscale KubeRay Connect installs three Anyscale components into your dataplane.
The Anyscale Connector is responsible for control operations: syncing state between the Anyscale Platform and your cluster, and serving a MutatingWebhook that patches KubeRay CRs at admission. It is a Helm-installed Deployment, running on a single pod as of now.
The Cluster Telemetry Gateway is responsible for aggregating logs and metrics from your Ray workloads and ships them to the Anyscale control plane. It is an OpenTelemetry Collector, deployed with a small set of sidecars.
The Observability API runs as a Deployment and serves the telemetry surface the Anyscale console reads from.
Splitting control from telemetry is a deliberate choice. The first benefit is fault tolerance – the Connector is only needed at the beginning of a pod's lifecycle, to mutate the spec; after that it can crash without affecting any metrics already flowing. The reverse holds too — a telemetry failure has no effect on admitting new workloads. This separation also yields both security (a split in identity that prevents bad actors) and scalability benefits (concurrent operations trigger Kubernetes autoscaling independently).
Four properties come out of this design:
Egress-only. Every connection is initiated from your dataplane. The Anyscale control plane needs no ingress into your cluster, ever.
Kubernetes-native authentication. The Connector authenticates with a projected service account OIDC token, verified against your cluster's own JWKS, removing the need for an Anyscale API key in the loop. This works on any conformant cluster whose OIDC issuer we can reach; for private clusters, an API key can be used for the initial Connector registration instead.
Resilient to control plane outages. Ray workloads keep running and keep making progress even if the Anyscale control plane is unavailable.
Thin-shim Connector. Control logic is pulled from the control plane rather than baked into the chart, so fixes and improvements can ship without you running a Helm upgrade.
The mutating webhook is fail-closed by default. If the Connector is unavailable when a KubeRay CR is submitted, the CR is rejected rather than admitted without telemetry wired up. We chose this over fail-open to avoid silently admitting workloads in an incomplete state. The behavior is configurable.
If the Connector fails after a workload is admitted, telemetry keeps flowing, the Gateway is independent of it. What you lose is the job state: the Connector is what reports phase transitions back to the platform, so a job moving from pending to running to terminated won't update in the console until the Connector recovers. The workload itself is unaffected.
LinkWhat you get on day one
LinkObservability
Once the telemetry gateway is flowing, you get the Anyscale observability out of the box for your KubeRay workloads, and it works whether you submit through kubectl apply or through the Anyscale API:
Ray workload dashboards — Ray Data, Ray Train, and Cluster views, built for Ray
Task and actor dashboards, which persist after the cluster terminates
Aggregated logs via Loki, plus per-workload logs
Workload, service, and job events, including autoscaler logs
Telescope, with a new full-stack view spanning your hardware to your Ray applications, across your fleet
That last one is worth highlighting: Telescope now correlates signals from the hardware layer (DCGM) through Kubernetes infrastructure events all the way up to your Ray applications — so a slow training job and the failing GPU underneath it show up in the same place. We wrote about that work in more detail in our recent full-stack observability post.
This is the observability tooling Anyscale has built into the platform, now available on KubeRay workloads in minutes.
LinkOrchestration
The preview also covers workload submission, scheduling, and status tracking on Kubernetes:
Submit KubeRay CRs directly to the cluster with
kubectl apply -f job.yaml, or through the platform withanyscale job submit -f job.yamlTrack job progress, status, and retries from the Anyscale console
Queue with Kueue, by applying Kueue configuration to your CRs
Queue with the Anyscale Scheduler, in Kueue passthrough or layered mode — coming soon
LinkWhat's next
The private preview covers onboarding, observability, and single-cluster orchestration. From here we are working on:
Multi-cluster orchestration. The Orchestrate tier, using Observe's health and availability signals to place workloads across clusters.
Identity passthrough. Mapping cluster identity to platform identity so RBAC, attribution, and quotas work on the cluster submission path.
For services, multi-region serving.
LinkGet started
Anyscale KubeRay Connect is in private preview. If you are running Ray on Kubernetes and want Anyscale without changing how you deploy, we would like to work with you.
