Product Documentation for Red Hat AI Inference Server 3.2

Highlights of what is new and what has changed with this Red Hat AI Inference Server release

Getting started with Red Hat AI Inference Server

Deploy Red Hat AI Inference Server in a disconnected environment using OpenShift Container Platform and a disconnected mirror image registry

Deploy Red Hat AI Inference Server in OpenShift Container Platform clusters that have supported AI accelerators installed

Supported hardware and software configurations for deploying Red Hat AI software

Red Hat AI validated models

Inferencing OCI-compliant models in Red Hat AI Inference Server

Server arguments for running Red Hat AI Inference Server

Compressing large language models with the LLM Compressor library

Configuring tool calling and chat templates for AI Inference Server

Find release notes and product documentation for using the OpenShift AI platform and its integrated MLOps capabilities to manage predictive and generative AI models at scale

Switch to the Red Hat Enterprise Linux AI documentation

Understand the product life cycle to plan deployments and support applications using the product

Explore a curated collection of learning resources designed to help you accomplish key tasks with Red Hat AI products and services