Ray Tune is a scalable machine learning library for hyperparameter tuning. Get started with Ray Tune on Anyscale today.

ML library for hyperparameter tuning. Anyscale supports and further optimizes Ray Tune for improved performance, reliability, and scale.

Ray Tune is a Python library for experiment execution and hyperparameter tuning at any scale. 

With Ray Tune, tune your favorite machine learning framework (scikit-learn, XGBoost, StatsForecast, and more) by running state of the art algorithms such as Population Based Training (PBT) and HyperBand. Tune also integrates with a wide range of additional hyperparameter optimization tools, including Ax, Optuna, and more.

Tune Map Small

Hyperparameter optimization helps you quickly increase your model performance. Take advantage of cutting-edge optimization algorithms to reduce the cost of fine-tuning and optimize training schedules.

Run your existing code in parallel across a cluster with just [a few code snippets](https://docs.ray.io/en/latest/tune/getting-started.html#tune-tutorial). Run and scale many trials in parallel, as easily as you coding on your laptop.

Ray Tune makes it easy to speed up hyperparameter searches by scaling out a Ray cluster. Speed up a single trial by running it across compute instances, or run more trials at a time by adding more nodes.

Ray Tune removes boilerplate from your code training workflow, supporting [logs results to tools](https://docs.ray.io/en/latest/tune/tutorials/tune-output.html#tune-logging) such as Weights & Biases, MLflow, and TensorBoard and [multiple storage options for experiment results](https://docs.ray.io/en/latest/tune/tutorials/tune-storage.html#tune-storage-options) (NFS, cloud storage), while also being highly customizable.

“We have no ceiling on scale, and an incredible opportunity to bring AI features and value to our 170 million users.”

Canva Logo Black

“Ray enables us to run deep learning workloads 12x faster, to reduce costs by 8x, and to train our models on 100x more data.”

Instacart Black

__Haixuin Wang__
VP of Engineering, Instacart

"What we’re doing would be prohibitively expensive with other AI platforms. Anyscale  enables us to quickly take advantage of open models like Llama-3, Mixtral, or whatever comes next, and in AI, there’s always a ‘next’. We’re ready for it with Anyscale."

Handshake

__Deepak Kumar__
VP of AI and Data, Handshake

Learn more about why Anyscale’s Ray Tune is the leader for distributed hyperparameter tuning at scale.


Explore in-depth documentation on how to get started and use Ray Tune.

Learn how to train many ML models without breaking the bank.

Learn how to use the Ray Tune library to distribute the hyperparameter tuning task among several computers. 

Get an overview of Ray Tune and demo of using it for tuning a deep learning model.

Manual hyperparameter tuning is slow and tedious. Automated hyperparameter tuning methods like grid search, random search, and Bayesian optimization can help us find optimal hyperparameters more quickly. Even so, tuning hyperparameters on a single computer can still take a long time. Using a single computer with limited resources (CPU and RAM) to try every hyperparameter combination the data set’s search algorithm produces creates a major bottleneck because each hyperparameter combination that the random search selects is independent of the previous and next selections.

To accelerate the search, Ray Tune runs different hyperparameter combinations in parallel on in the cloud. By distributing the hyperparameter tuning task among several computers, we can increase the speed and effectiveness of hyperparameter tuning

In principle, this isn’t an easy task. It usually requires significant code refactoring. However, Ray provides the means to build distributed applications with few code changes, making it quick and easy to do hyperparameter tuning at scale.


Anyscale is the smartest place to host Ray, providing a managed experience that increases performance, optimizes utilization, and reduces costs. Anyscale improves Ray Tune with advanced features like: 
- Fast autoscaling (launch 1000+ nodes in 60 seconds or less)
- Integrated workspace with VSCode and Jupyter Notebooks
- Advanced checkpointing and fault tolerance
- Alerting
- And much more!


Nope! In addition to Ray Core and Ray Data, we’ve also developed the following open source libraries: 
- [Ray Data](https://www.anyscale.com/product/library/ray-data)
- [Ray Train](https://anyscale.com/product/library/ray-train)
- [Ray Serve](https://anyscale.com/product/library/ray-serve)
- [RLlib (Reinforcement learning)](https://anyscale.com/product/library/ray-rllib)

As well as the following Anyscale only libraries: 
- [LLMForge](https://anyscale.com/product/library/llmforge) (for LLM fine-tuning)
- [RayLLM](https://anyscale.com/product/library/ray-llm) (for LLM inference) 


Ray has a robust distributed dataplane, combining decentralized scheduling with a best-in-class distributed object layer, featuring:
- __Efficient zero-copy reads__ via shared memory for workers on the same node, obviating the need for serialization when sharing data across worker processes.
- __Locality-aware scheduling__, where data-intensive tasks are scheduled onto the node that has the most of the task’s required data already local.
- __Resilient object transfer protocols__, with a memory manager that ensures prefetching and forward progress while bounding the amount of memory usage.
- __A fast data transport implementation__, transferring data chunks in parallel in order to maximize transfer throughput.

Ray Tune builds on the existing Ray dataplane to provide additional higher-level features for hyperparameter tuning.


There are a variety of popular open source repositories and research projects that leverage Tune. Feel free to submit a pull-request adding (or requesting a removal!) of a listed project.
- [Softlearning](https://github.com/rail-berkeley/softlearning): Softlearning is a reinforcement learning framework for training maximum entropy policies in continuous domains. Includes the official implementation of the Soft Actor-Critic algorithm.
- [Flambe](https://github.com/asappresearch/flambe): An ML framework to accelerate research and its path to production. See [flambe.ai](https://flambe.ai/).
- [Population Based Augmentation](https://github.com/arcelien/pba): Population Based Augmentation (PBA) is a algorithm that quickly and efficiently learns data augmentation functions for neural network training. PBA matches state-of-the-art results on CIFAR with one thousand times less compute.
- [Fast AutoAugment by Kakao](https://github.com/kakaobrain/fast-autoaugment): Fast AutoAugment (Accepted at NeurIPS 2019) learns augmentation policies using a more efficient search strategy based on density matching.
- [Allentune](https://github.com/allenai/allentune): Hyperparameter Search for AllenNLP from AllenAI.
- [machinable](https://github.com/frthjf/machinable): A modular configuration system for machine learning research. See [machinable.org](https://machinable.org/).
- [NeuroCard](https://github.com/neurocard/neurocard): NeuroCard (Accepted at VLDB 2021) is a neural cardinality estimator for multi-table join queries. It uses state of the art deep density models to learn correlations across relational database tables.

If you’re already using an existing hyperparameter tuning tool such as HyperOpt or Bayesian Optimization, Ray Tune actually allows you to power up your existing workflow. Tune’s [Search Algorithms](https://docs.ray.io/en/latest/tune/api/suggestion.html#tune-search-alg) integrate with a variety of popular hyperparameter tuning libraries (see [examples](https://docs.ray.io/en/latest/tune/examples/index.html#tune-examples-ref)) and allow you to seamlessly scale up your optimization process—without sacrificing performance.

Anyscale offers a usage based consumption model. Pay for only what you use.  

Learn more on the [Anyscale Pricing](https://www.anyscale.com/pricing-detail?utm_source=anyscale&utm_content=library) page.

Build, deploy, and manage scalable AI and Python applications on the leading AI platform. Unlock your AI potential with Anyscale.

Ray Tune with Anyscale

Ray Tune

Visit Anyscale at Microsoft Build, Booth G201, June 2-3: 

Powered by Ray, Anyscale empowers AI builders to run and scale all ML and AI workloads on any cloud and on-prem.

Ray Tune

What is Ray Tune?

Benefits

Cutting-Edge Optimization Algorithms

First-Class Developer Productivity

Multi-GPU and Distributed Training

Effortlessly Integrate With Your Toolkit

FAQs

Try Anyscale Today

Ray Tune

What is Ray Tune?

Benefits

Cutting-Edge Optimization Algorithms

First-Class Developer Productivity

Multi-GPU and Distributed Training

Effortlessly Integrate With Your Toolkit

Related Resources

Ray Tune Docs

Training 1M ML Models in Record Time with Ray

How to Distribute Hyperparameter Tuning Using Ray Tune

Webinar: Fast & Efficient Hyperparameter Tuning with Ray Tune

FAQs

Why Use Ray Tune?

What’s the difference between OSS Ray Tune and Anyscale’s Ray Tune?

Is Ray Tune the only Ray library?

Why is Ray Tune built on top of Ray Core?

How is Ray Tune used by others?

What if I’m already using another hyperparameter tuning framework?

How much does Anyscale cost?

Try Anyscale Today