DEVOPSdigest

Red Hat announced the latest version of Red Hat OpenShift AI, its artificial intelligence (AI) and machine learning (ML) platform built on Red Hat OpenShift that enables enterprises to create and deliver AI-enabled applications at scale across the hybrid cloud.

Red Hat OpenShift AI 2.15 is designed to provide greater flexibility, tuning and tracking capabilities, helping to accelerate enterprises’ AI/ML innovation and operational consistency with greater regularity and a stronger security posture at scale across public clouds, datacenters and edge environments.

Red Hat OpenShift AI 2.15 is aimed to help enterprises address the emerging needs of AI workloads alongside the requirements of the mission-critical, cloud-native applications that power their businesses today.

The advanced features delivered with the latest version of Red Hat OpenShift AI include:

- Model registry, currently provided as technology preview, is the central place to view and manage registered models. This feature provides a structured and organized way to share, version, deploy and track predictive and gen AI models, metadata and model artifacts. The option to provide multiple model registries is also available. Red Hat has also donated the model registry project to the Kubeflow community as a subproject.

- Data drift detection monitors changes in input data distributions for deployed ML models. This feature allows data scientists to detect when the live data used for model interference significantly deviates from the data upon which the model was trained. Drift detection helps verify model reliability by continuously monitoring input data, keeping the model aligned with real-world data and helping to maintain the accuracy of its predictions over time.

- Bias detection tools help data scientists and AI engineers monitor whether their models are fair and unbiased, a crucial part of establishing model trust. These tools not only help to provide insights on whether models are unbiased based on the training data but also monitor these models for fairness during real-world deployments. These tools are incorporated from the TrustyAI open source community, which provides a diverse toolkit for responsible AI development and deployment.

- Efficient fine-tuning with LoRA uses low-rank adapters (LoRA) to enable more efficient fine-tuning of LLMs, such as Llama 3. This allows organizations to scale AI workloads while reducing costs and resource consumption. By optimizing model training and fine-tuning within cloud native environments, this solution enhances both performance and flexibility, making AI deployment more accessible and scalable.

- Support for NVIDIA NIM, a set of easy-to-use interface microservices that accelerate the delivery of gen AI applications. The integration with NIM, part of the NVIDIA AI Enterprise software platform, will help speed up gen AI deployments while supporting a wide range of AI models to deliver scalable inference on-premises or in the cloud through application programming interfaces (APIs).

- Support for AMD GPUs enables access to an AMD ROCm workbench image for using AMD GPUs for model development. The new functionality also enables access to images that can be used for serving and training/tuning use cases with AMD GPUs. This support gives organizations additional options for using GPUs to improve performance for computationally intensive activities.

As a comprehensive AI/ML platform, Red Hat OpenShift AI 2.15 also adds new capabilities around gen AI model serving, including vLLM serving runtime for KServe. This new capability brings the popular open source model serving runtime for large language models (LLMs) to the platform. The flexibility and performance of vLLM is an excellent addition to the platform’s currently supported runtimes, with users also able to add their own custom options as business requirements dictate.

The latest version of Red Hat OpenShift AI also adds support for KServe Modelcars, which adds Open Container Initiative (OCI) repositories as an option for storing and accessing containerized model versions. Additionally, private/public route selection for endpoints in KServe enables organizations to enhance the security posture of a model by directing it specifically to internal endpoints when required.
Expanded AI training and experimentation options

Red Hat OpenShift AI 2.15 adds enhancements to data science pipelines and experiment tracking, empowering data scientists to more easily manage, compare and analyze pipeline runs grouped in a logical structure. The platform also adds hyperparameter tuning with Ray Tune, adding advanced optimization algorithms to improve the accuracy and train predictive and gen AI models more efficiently. The base container images for Ray clusters are now included in the newest version of Red Hat OpenShift AI, and training and tuning jobs can be scheduled across distributed workloads in the cluster to speed up jobs and maximize node utilization.

Red Hat OpenShift 2.15 will be generally available beginning mid-November 2024.