Release notes

Red Hat Enterprise Linux AI 3.5

Red Hat Enterprise Linux AI release notes

Red Hat RHEL AI Documentation Team

Abstract

Review new features, enhancements, resolved issues, and known issues associated with this release.

Preface

Red Hat Enterprise Linux AI provides developers and IT organizations with a scalable inference platform for deploying and customizing AI models on secure, scalable resources with minimal configuration and resource usage.

Chapter 1. Version 3.5 release notes

Red Hat Enterprise Linux AI is a generative AI inference platform for Linux environments that uses Red Hat AI Inference for running and optimizing models, and includes Red Hat AI Model Optimization Toolkit for model quantization, sparsity, and general compression for supported AI accelerators. Red Hat AI Model Optimization Toolkit has native Hugging Face and vLLM support. You can seamlessly integrate optimized models with deployment pipelines for faster, cost-saving inference at scale, powered by the compressed-tensors model format.

Red Hat Enterprise Linux AI is packaged as bootc container images for easy deployment on a Linux server appliance with NVIDIA CUDA, AMD ROCm, or Intel Gaudi AI accelerators installed. Download the images from registry.redhat.io.

The following bootc and disk-image container images are Generally Available (GA):

  • registry.redhat.io/rhelai3/bootc-aws-cuda-rhel9:3.5.0
  • registry.redhat.io/rhelai3/bootc-azure-cuda-rhel9:3.5.0
  • registry.redhat.io/rhelai3/bootc-azure-rocm-rhel9:3.5.0
  • registry.redhat.io/rhelai3/bootc-cuda-rhel9:3.5.0
  • registry.redhat.io/rhelai3/bootc-gcp-cuda-rhel9:3.5.0
  • registry.redhat.io/rhelai3/bootc-rocm-rhel9:3.5.0
  • registry.redhat.io/rhelai3/disk-image-cuda-rhel9:3.5.0

The following bootc container image is Technology Preview:

  • registry.redhat.io/rhelai3/bootc-gaudi-rhel9:3.5.0
Important

There is no direct upgrade path from Red Hat Enterprise Linux AI 1.5 to Red Hat Enterprise Linux AI 3.5. You can upgrade from Red Hat Enterprise Linux AI 3.0 to 3.5 and all versions in-between.

Important

The registry.redhat.io/rhelai3/bootc-rocm-rhel9:3.5.0 image does not include Red Hat AI Model Optimization Toolkit, which is not supported for AMD ROCm AI accelerators.

Additional resources

1.1. New features

Red Hat Enterprise Linux AI 3.5 packages Red Hat AI Inference 3.5.0, which includes the following highlights:

Upgraded vLLM to v0.24.0
Red Hat AI Inference 3.5.0 packages the upstream vLLM v0.24.0 release. You can review the complete list of updates in the upstream vLLM v0.24.0 release notes.
Red Hat AI Model Optimization Toolkit updated to v0.12.0
Red Hat AI Model Optimization Toolkit 3.5.0 packages the upstream LLM Compressor v0.12.0 release. You can review the complete list of updates in the upstream LLM Compressor v0.12.0 release notes.
Accelerator software stack updates

Red Hat Enterprise Linux AI 3.5 updates accelerator software components as follows:

  • AMD ROCm updated from 7.1.1 to 7.14 (TheRock framework)
  • NVIDIA CUDA toolkit updated from 13.0.1 to 13.0.2
  • Intel Gaudi software stack (Tech Preview) updated from 1.23.0 to 1.24.1-482
Note

For CUDA 13.0.2, the minimum supported NVIDIA driver version is 580.173.02. The packaged NVIDIA driver version for this release can change as long as it remains compatible with CUDA 13.0.2.

For the complete list of new features, enhancements, and known issues, see the Red Hat AI Inference 3.5.0 release notes.

Additional resources

1.2. Known issues

NVIDIA HGX B200 and B300 servers require additional NVLink packages

The Red Hat Enterprise Linux AI bootc image does not currently include the OFED or MOFED InfiniBand drivers that NVIDIA Fabric Manager requires on HGX B200 and B300 NVSwitch-based systems. Without these drivers, Fabric Manager fails to start and GPU initialization fails with a cudaGetDeviceCount() error (Error 802: system not yet initialized).

As a workaround, you can disable NVLink by running the following command and rebooting the system: 

# echo "options nvidia NVreg_NvLinkDisable=1" >> /etc/modprobe.d/nvidia.conf
Important

Disabling NVLink forces GPU-to-GPU communication over PCIe, which significantly reduces multi-GPU bandwidth. Expect higher latency and lower throughput for models that span multiple GPUs.

Additional resources

Chapter 2. Version 3.5.0-ea.2 release notes

Red Hat Enterprise Linux AI is a generative AI inference platform for Linux environments that uses Red Hat AI Inference for running and optimizing models, and includes Red Hat AI Model Optimization Toolkit for model quantization, sparsity, and general compression for supported AI accelerators.

Important

Red Hat Enterprise Linux AI 3.5.0-ea.2 is an Early Access release. Early Access releases are not supported by Red Hat in any way and are not functionally complete or production-ready. Do not use Early Access releases for production or business-critical workloads. Use Early Access releases to test upcoming product features in advance of their possible inclusion in a Red Hat product offering, and to test functionality and provide feedback during the development process. These features might not have any documentation, are subject to change or removal at any time, and testing is limited. Red Hat might provide ways to submit feedback on Early Access features without an associated SLA.

Red Hat Enterprise Linux AI is packaged as a bootc container image for easy deployment on a Linux server appliance with NVIDIA CUDA, AMD ROCm, or Intel Gaudi AI accelerators installed. The following container images are available as early access releases from Content from registry.redhat.io is not included.registry.redhat.io:

  • registry.redhat.io/rhelai-early-access/bootc-aws-cuda-rhel9:3.5.0-ea.2
  • registry.redhat.io/rhelai-early-access/bootc-azure-cuda-rhel9:3.5.0-ea.2
  • registry.redhat.io/rhelai-early-access/bootc-azure-rocm-rhel9:3.5.0-ea.2
  • registry.redhat.io/rhelai-early-access/bootc-cuda-rhel9:3.5.0-ea.2
  • registry.redhat.io/rhelai-early-access/bootc-gaudi-rhel9:3.5.0-ea.2
  • registry.redhat.io/rhelai-early-access/bootc-gcp-cuda-rhel9:3.5.0-ea.2
  • registry.redhat.io/rhelai-early-access/bootc-rocm-rhel9:3.5.0-ea.2

A Red Hat Enterprise Linux AI CUDA disk image is also available:

  • registry.redhat.io/rhelai-early-access/disk-image-cuda-rhel9:3.5.0-ea.2
Important

There is no direct upgrade path from Red Hat Enterprise Linux AI 1.5 to Red Hat Enterprise Linux AI 3.0. You can upgrade from Red Hat Enterprise Linux AI 3.0 to 3.5 and all versions in-between.

Important

The registry.redhat.io/rhelai-early-access/bootc-rocm-rhel9:3.5.0-ea.2 image does not include Red Hat AI Model Optimization Toolkit, which is not supported for AMD ROCm AI accelerators.

Additional resources

2.1. New features

Red Hat Enterprise Linux AI 3.5.0-ea.2 packages Red Hat AI Inference 3.5.0-ea.2, which includes the following highlights:

Upgraded vLLM to v0.21.0
Red Hat AI Inference 3.5.0-ea.2 packages the upstream vLLM v0.21.0 release. You can review the complete list of updates in the upstream Content from github.com is not included.vLLM v0.21.0 release notes.
AMD MI350P PCIe support
Red Hat AI Inference 3.5.0-ea.2 adds support for AMD MI350P PCIe accelerators with ROCm.
Updated Red Hat AI Model Optimization Toolkit
Red Hat AI Model Optimization Toolkit 3.5.0-ea.2 packages the upstream LLM Compressor v0.10.0.2 release. You can review the complete list of updates in the upstream Content from github.com is not included.LLM Compressor v0.10.0.2 release notes.

2.2. Known issues

There are no known issues for Red Hat Enterprise Linux AI 3.5.0-ea.2.

Note

Review the Red Hat AI Inference 3.5.0-ea.2 known issues for additional issues that might impact your Red Hat Enterprise Linux AI environment.

Chapter 3. Version 3.5.0-ea.1 release notes

Red Hat Enterprise Linux AI is a generative AI inference platform for Linux environments that uses {rhaiis} for running and optimizing models, and includes Red Hat AI Model Optimization Toolkit for model quantization, sparsity, and general compression for supported AI accelerators.

Important

Red Hat Enterprise Linux AI 3.5.0-ea.1 is an Early Access release. Early Access releases are not supported by Red Hat in any way and are not functionally complete or production-ready. Do not use Early Access releases for production or business-critical workloads. Use Early Access releases to test upcoming product features in advance of their possible inclusion in a Red Hat product offering, and to test functionality and provide feedback during the development process. These features might not have any documentation, are subject to change or removal at any time, and testing is limited. Red Hat might provide ways to submit feedback on Early Access features without an associated SLA.

Red Hat Enterprise Linux AI is packaged as a bootc container image for easy deployment on a Linux server appliance with NVIDIA CUDA or AMD ROCm AI accelerators installed. The following container images are available as early access releases from Content from registry.redhat.io is not included.registry.redhat.io:

  • registry.redhat.io/rhelai-early-access/bootc-aws-cuda-rhel9:3.5.0-ea.2
  • registry.redhat.io/rhelai-early-access/bootc-azure-cuda-rhel9:3.5.0-ea.2
  • registry.redhat.io/rhelai-early-access/bootc-azure-rocm-rhel9:3.5.0-ea.2
  • registry.redhat.io/rhelai-early-access/bootc-cuda-rhel9:3.5.0-ea.2
  • registry.redhat.io/rhelai-early-access/bootc-gaudi-rhel9:3.5.0-ea.2
  • registry.redhat.io/rhelai-early-access/bootc-gcp-cuda-rhel9:3.5.0-ea.2
  • registry.redhat.io/rhelai-early-access/bootc-rocm-rhel9:3.5.0-ea.2

A Red Hat Enterprise Linux AI CUDA disk image is also available:

  • registry.redhat.io/rhelai-early-access/disk-image-cuda-rhel9:3.5.0-ea.2
Important

There is no direct upgrade path from Red Hat Enterprise Linux AI 1.5 to Red Hat Enterprise Linux AI 3.0. You can upgrade from Red Hat Enterprise Linux AI 3.0 to 3.5 and all versions in-between.

Important

The registry.redhat.io/rhelai-early-access/bootc-rocm-rhel9:3.5.0-ea.2 image does not include Red Hat AI Model Optimization Toolkit, which is not supported for AMD ROCm AI accelerators.

Additional resources

3.1. New features

Red Hat Enterprise Linux AI 3.5.0-ea.1 packages {rhaiis} 3.5.0-ea.1, which includes the following highlights:

Upgraded vLLM to v0.19.1
{rhaiis} 3.5.0-ea.1 packages the upstream vLLM v0.19.1 release. You can review the complete list of updates in the upstream Content from github.com is not included.vLLM v0.19.1 release notes.
Gemma 4 model support
Added support for the RedHatAI/Gemma-4-26B-IT and RedHatAI/Gemma-4-12B-IT models. To use custom tool calling, add the template from the upstream vLLM repository or use the chat template from the Hugging Face model repository.
New model support
Added support for Mistral-small-3.2-24B-instruct-2506, multilingual-e5-large, multilingual-e5-large-instruct, and granite-4-8b-dense model architectures.
Custom weights support for Llama 3.1
Added support for Llama-3.1-8b-instruct, including fine-tuned models and custom weights.
Model support matrix

This release adds a comprehensive model support matrix to the documentation. Validated models are benchmarked with GuideLLM and LM Evaluation Harness. Enabled models are architecturally supported, but not yet through the full validation pipeline. The matrix lists supported models detailing the following information:

  • Validation status
  • Minimum RHEL AI platform version
  • vLLM version
  • GPU requirements, including minimum vRAM and supported GPU configurations
Speculators library update (Technology Preview)

Updated the Speculators library to version 0.5.0a0.

Important

Speculators library is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

Updated Red Hat AI Model Optimization Toolkit
Red Hat AI Model Optimization Toolkit 3.5.0-ea.1 packages the upstream LLM Compressor v0.10.0.2 release. You can review the complete list of updates in the upstream Content from github.com is not included.LLM Compressor v0.10.0.2 release notes.

3.2. Known issues

There are no known issues for Red Hat Enterprise Linux AI 3.5.0-ea.1.

Note

Review the Red Hat AI Inference 3.5.0-ea.1 known issues for issues that might impact your Red Hat Enterprise Linux AI environment.

Legal Notice

Copyright © Red Hat.
Except as otherwise noted below, the text of and illustrations in this documentation are licensed by Red Hat under the Creative Commons Attribution–Share Alike 3.0 Unported license . If you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, the Red Hat logo, JBoss, Hibernate, and RHCE are trademarks or registered trademarks of Red Hat, LLC. or its subsidiaries in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
XFS is a trademark or registered trademark of Hewlett Packard Enterprise Development LP or its subsidiaries in the United States and other countries.
The OpenStack® Word Mark and OpenStack logo are trademarks or registered trademarks of the Linux Foundation, used under license.
All other trademarks are the property of their respective owners.