You can discover and select an existing Kubernetes Gateway resource from the model serving UI when deploying LLMInferenceService models. This Technology Preview feature enables self-service Gateway management, supports multitenant namespace-scoped network isolation, and provides programmatic access through the Gateway discovery REST API.

Gateway discovery and creation is disabled by default and must be explicitly enabled by a cluster administrator through the dashboard configuration.

You can now deploy models using the vLLM runtime through Models-as-a-Service (MaaS). This Technology Preview feature enables you to serve large language models with vLLM’s high-performance inference capabilities while benefiting from MaaS governance and subscription-based controls.

Key capabilities include:

You can now configure Models-as-a-Service to authenticate users with an external OpenID Connect (OIDC) identity provider. This Technology Preview feature enables enterprise-wide access to large language models without requiring OpenShift accounts for every user.

Key capabilities include:

As a cluster administrator, you can now monitor platform-wide Models-as-a-Service (MaaS) usage through a dedicated observability dashboard embedded in the OpenShift AI console. This Technology Preview feature provides comprehensive usage metrics for cost attribution and showback reporting to finance teams.

Key capabilities include:

You can now route model inference requests to external model providers through the Models-as-a-Service (MaaS) inference gateway. This Technology Preview feature enables you to apply MaaS governance policies, token tracking, and rate limiting to third-party LLM services such as OpenAI, Anthropic, or other external providers.

Key capabilities include:

Red Hat OpenShift AI now provides recommended vLLM runtime configurations for select high-demand models. These configurations are designed to help users achieve maximum performance on specific hardware profiles, matching or exceeding industry benchmarks for low-latency and high-throughput serving.

Previously, users had to manually tune vLLM arguments to find the sweet spot for performance. Now, validated YAML templates are embedded directly in the model card Readme section as markdown, serving as a reference for manual configuration.

Key features of this update include:

Model registry now provides cryptographic signing and verification of AI artifacts by using the Python client API. This Technology Preview feature enables you to sign artifacts to establish authenticity and verify signatures to confirm integrity. Artifact signing provides foundational capabilities for model serving verification and AI asset provenance tracking in future releases.

For more information, see the Content from github.com is not included.Kubeflow Model Registry Python client documentation and Content from github.com is not included.Kubeflow async upload job documentation.

Red Hat OpenShift AI includes a dedicated suite of text embedding models within the Model Catalog that can be deployed on vLLM. This Technology Preview feature allows data scientists and AI engineers to discover and deploy models designed specifically for vector generation, a critical component for Retrieval-Augmented Generation (RAG) and semantic search workflows. By separating these models from standard generative large language models (LLMs), OpenShift AI provides a streamlined experience for users building vector-generation engines.

Key features of this update include:

To view embedding models in the Catalog, navigate to AI hub → Models → Catalog in the OpenShift AI dashboard, select the Other models category, and look for the text-embedding tag on model cards. You can deploy a model directly from its card by following the deployment wizard to configure the serving runtime and hardware profile. Once deployed, the model provides an API endpoint that generates numerical embeddings for text inputs for use in downstream vector databases or RAG pipelines.

The model deployment wizard includes a YAML viewer that provides real-time YAML preview and manual editing capabilities for LLMInferenceService deployments. You can toggle between Form and YAML views to see the generated LLMInferenceService YAML as you fill out the deployment form. The preview updates automatically as form fields change, enabling you to verify configuration before deployment. You can also enter Manual Edit Mode to edit YAML directly for advanced Distributed Inference with llm-d parameters not exposed in the form, such as autoscaling, disaggregated serving, LoRA adapters, and custom runtime configurations.

This Technology Preview feature introduces the following capabilities:

Red Hat OpenShift AI 3.5 EA1 updates the guardrails experience in the Gen AI Studio playground. Guardrails are now powered by NeMo Guardrails, providing more reliable and consistent safety checks for model interactions.

You can independently control guardrails for user input and model output directly from the playground settings:

The OpenAI Conversations API is now available on OGX as a Technology Preview in OpenShift AI 3.5 EA1. This development tool allows you to build context-aware, multi-turn AI applications.

Key capabilities of the Conversations API include:

You can compare results across multiple configurations in the Playground by using multiple chat panes side-by-side. Each pane supports unique configurations for models, prompts, MCP servers, guardrails, and knowledge sources. You can run synchronized prompts across all panes or use independent prompts for each pane.

Key capabilities include:

The Gen AI Playground provides access to basic safety guardrails from OGX. You can enable or disable guardrails in the playground interface to filter unsafe content and detect prompt injection attempts.

The following guardrails are available:

Starting with OpenShift AI 3.3, OGX provides OpenAI-compatible grounding and citation annotations for search-backed responses as a Technology Preview feature.

This enhancement enables retrieval-augmented generation (RAG) applications to trace generated responses back to source documents by using the same annotation schemas returned by OpenAI Search and Responses APIs. The feature supports document source attribution and preserves citation metadata in API responses, allowing existing OpenAI client applications to consume citation information without code changes.

The OGX Operator in OpenShift AI 3.3 now offers ConfigMap driven image updates for OGXServer resources. This allows you to patch security or bug fixes without new Operator versions. To enable this feature, update your ConfigMap with the following parameters:

 image-overrides: |
    starter-gpu: registry.redhat.io/rhoai/odh-ogx-core-rhel9:v3.3
    starter: registry.redhat.io/rhoai/odh-ogx-core-rhel9:v3.3

Using the starter-gpu and starter distributions names as the key allows the operator to apply these overrides automatically.

To update the OGX Distributions image for all starter distributions, run the following command:

$ kubectl patch configmap ogx-operator-config -n ogx-k8s-operator-system --type merge -p '{"data":{"image-overrides":"starter: quay.io/opendatahub/ogx:latest"}}'

This allows the OGXServer resources to restart with the new image.

This feature is available as a Technology Preview.

OpenShift AI now includes Model-as-a-Service (MaaS) to address resource consumption and governance challenges associated with serving large language models (LLMs).

MaaS provides centralized control over model access and resource usage by exposing models through managed API endpoints, allowing administrators to enforce consumption policies across teams.

This Technology Preview introduces the following capabilities:

The MLServer serving runtime for KServe is now available as a technology preview feature in Red Hat OpenShift AI. You can use this runtime to deploy models trained on structured data, such as classical machine learning models. You can deploy models directly without converting them to ONNX format, which simplifies the deployment process and improves performance.

This feature provides support for the following common machine learning frameworks:

Starting with OpenShift AI 3.2, you can use PostgreSQL with the pgvector extension as a remote vector store provider for the OGX vector_store endpoint as a Technology Preview feature.

This enhancement enables vector storage backed by PostgreSQL, providing durable and transactional persistence for vector embeddings. For more information, see OGX API provider support and Deploying a PostgreSQL instance with pgvector.

OpenShift AI 3.2 allows you to customize your embedding models as a Technology Preview feature. In the version of OGX shipped in OpenShift AI 3.2, vLLM controls embeddings by default. You can update the VLLM_EMBEDDING_URL environment variable in your OGXServer custom resource to enable embeddings, or you can use custom embeddings providers. For example:

  - name: ENABLE_SENTENCE_TRANSFORMERS
    value: "true"
  - name: EMBEDDING_PROVIDER
    value: "sentence-transformers"

Kubeflow Trainer v2 is now available as a Technology Preview feature in OpenShift AI 3.2.

Kubeflow Trainer v2 is the next generation of distributed training for OpenShift AI, replacing the Kubeflow Training Operator v1 (KFTOv1). This Kubernetes-native solution simplifies how data scientists and ML engineers run PyTorch training workloads at scale using a unified TrainJob API and Python SDK.

This Technology Preview release introduces the following capabilities:

You can now use the Guardrails Orchestrator from TrustyAI with OGX as a Technology Preview feature.

This integration enables built-in detection and evaluation workflows to support AI safety and content moderation. When TrustyAI is enabled and the FMS Orchestrator and detectors are configured, no manual setup is required.

To activate this feature, set the following field in the DataScienceCluster custom resource for the OpenShift AI Operator: spec.ogx.managementState: Managed

For more information, see the TrustyAI FMS Provider on GitHub: Content from github.com is not included.TrustyAI FMS Provider.

A new AI Available Assets page enables AI engineers and application developers to view and consume deployed AI resources within their projects.

This enhancement introduces a filterable UI that lists available models and Model Context Protocol (MCP) servers in the selected project, allowing users with appropriate permissions to identify accessible endpoints and integrate them directly into the AI Playground or other applications.

The Generative AI (GenAI) Playground introduces a unified, interactive experience within the OpenShift AI dashboard for experimenting with foundation and custom models.

Users can test prompts, compare models, and evaluate Retrieval-Augmented Generation (RAG) workflows by uploading documents and chatting with their content. The GenAI Playground also supports integration with approved Model Context Protocol (MCP) servers and enables export of prompts and agent configurations as runnable code for continued iteration in local IDEs.

Chat context is preserved within each session, providing a suitable environment for prompt engineering and model experimentation.

You can now install and operate OGX and RAG/Agentic components in fully disconnected (air-gapped) OpenShift AI environments.

This enhancement enables secure deployment of OGX features without internet access, allowing organizations to use AI capabilities while maintaining compliance with strict network security policies.

This feature is available as a Technology Preview.

The Feature Store is now integrated with OpenShift AI, data science projects, and workbenches. This integration also introduces centrally managed, role-based access control (RBAC) capabilities for improved governance.

These enhancements provide two key capabilities:

The Feature Store component now includes a web-based user interface (UI).

You can use the UI to view registered Feature Store objects and their relationships, such as features, data sources, entities, and feature services.

To enable the UI, edit your FeatureStore custom resource (CR) instance. When you save the change, the Feature Store Operator starts the UI container and creates an OpenShift route for access.

For more information, see Setting up the Feature Store user interface for initial use.

With this release, the OGX Technology Preview feature enables Retrieval-Augmented Generation (RAG) and agentic workflows for building next-generation generative AI applications. It supports remote inference, built-in embeddings, and vector database operations. It also integrates with providers like TrustyAI’s provider for safety and Trusty AI’s LM-Eval provider for evaluation.

This preview includes tools, components, and guidance for enabling the OGX Operator, interacting with the RAG Tool, and automating PDF ingestion and keyword search capabilities to enhance document discovery.

Centralized platform observability, including metrics, traces, and built-in alerts, is available as a Technology Preview feature. This solution introduces a dedicated, pre-configured observability stack for OpenShift AI that allows cluster administrators to perform the following actions:

The OGX Distribution now includes version 0.3.0 as a Technology Preview feature.

This update introduces several enhancements, including expanded support for retrieval-augmented generation (RAG) pipelines, improved evaluation provider integration, and updated APIs for agent and vector store management. It also provides compatibility updates aligned with recent OpenAI API extensions and infrastructure optimizations for distributed inference.

The previously supported version was 0.2.22.

OpenShift AI now supports Kubernetes Event-driven Autoscaling (KEDA) in its KServe RawDeployment mode. This Technology Preview feature enables metrics-based autoscaling for inference services, allowing for more efficient management of accelerator resources, reduced operational costs, and improved performance for your inference services.

To set up autoscaling for your inference service in KServe RawDeployment mode, you need to install and configure the OpenShift Custom Metrics Autoscaler (CMA), which is based on KEDA.

For more information about this feature, see: This content is not included.Configuring metrics-based autoscaling.

You can now create and manage Ray Jobs on Ray Clusters directly through the CodeFlare SDK.

This enhancement aligns the CodeFlare SDK workflow with the KubernetesFlow Training Operator (KFTO) model, where a job is created, run, and completed automatically. This enhancement simplifies manual cluster management by preventing Ray Clusters from remaining active after job completion.

You can now use a custom flow estimator for synthetic data generation (SDG) pipelines.

For supported and compatible tagged SDG teacher models, the estimator helps you evaluate a chosen teacher model, custom flow, and supported hardware on a sample dataset before running full workloads.

OGX core can now deploy and run efficiently on single node OpenShift (SNO).

This enhancement optimizes component startup and resource usage so that OGX can operate reliably in single-node cluster environments.

You can now use the FAISS (Facebook AI Similarity Search) library as an inline vector store in OpenShift AI.

FAISS is an open-source framework for high-performance vector search and clustering, optimized for dense numerical embeddings with both CPU and GPU support. When enabled with an embedded SQLite backend in the OGX Distribution, FAISS stores embeddings locally within the container, removing the need for an external vector database service.

You can now install and manage Feature Store as a configurable component in OpenShift AI. Based on the open-source Content from feast.dev is not included.Feast project, Feature Store acts as a bridge between ML models and data, enabling consistent and scalable feature management across the ML lifecycle.

This Technology Preview release introduces the following capabilities:

You can now deploy OGX and RAG or agentic solutions in regulated environments that require FIPS compliance.

This enhancement provides FIPS-certified and compatible deployment patterns to help organizations meet strict regulatory and certification requirements for AI workloads.

Validated sdg_hub example notebooks are now available to provide a notebook-driven user experience in OpenShift AI 3.0.

These notebooks support multiple Red Hat platforms and enable customization through SDG pipelines. They include examples for the following use cases:

You can now use the Retrieval-Augmented Generation Assessment (RAGAS) evaluation provider to measure the quality and reliability of RAG systems in OpenShift AI.

RAGAS provides metrics for retrieval quality, answer relevance, and factual consistency, helping you identify issues and optimize RAG pipeline configurations.

The integration with the OGX evaluation API supports two deployment modes:

The hardware profiles feature enables users to target specific worker nodes for workbenches or model-serving workloads. It allows users to target specific accelerator types or CPU-only nodes.

This feature replaces the current accelerator profiles feature and container size selector field, offering a broader set of capabilities for targeting different hardware configurations. While accelerator profiles, taints, and tolerations provide some capabilities for matching workloads to hardware, they do not ensure that workloads land on specific nodes, especially if some nodes lack the appropriate taints.

The hardware profiles feature supports both accelerator and CPU-only configurations, along with node selectors, to enhance targeting capabilities for specific worker nodes. Administrators can configure hardware profiles in the settings menu. Users can select the enabled profiles using the UI for workbenches, model serving, and AI pipelines where applicable.

With the RStudio Server workbench image, you can access the RStudio IDE, an integrated development environment for R. The R programming language is used for statistical computing and graphics to support data analysis and predictions.

To use the RStudio Server workbench image, you must first build it by creating a secret and triggering the BuildConfig, and then enable it in the OpenShift AI UI by editing the rstudio-rhel9 image stream. For more information, see Building the RStudio Server workbench images.

With the CUDA - RStudio Server workbench image, you can access the RStudio IDE and NVIDIA CUDA Toolkit. The RStudio IDE is an integrated development environment for the R programming language for statistical computing and graphics. With the NVIDIA CUDA toolkit, you can enhance your work by using GPU-accelerated libraries and optimization tools.

To use the CUDA - RStudio Server workbench image, you must first build it by creating a secret and triggering the BuildConfig, and then enable it in the OpenShift AI UI by editing the rstudio-rhel9 image stream. For more information, see Building the RStudio Server workbench images.

You can surface previously-created vector stores as retrieval-augmented generation (RAG) knowledge sources in the Gen AI Studio Playground. Platform engineers define vector stores through ConfigMaps, and AI engineers can select them as knowledge sources when chatting with models in the Playground. This enables rapid RAG experimentation without writing code or re-ingesting data.

This release includes the following capabilities:

Red Hat OpenShift AI provides an MCP (Model Context Protocol) server that enables MCP-compatible clients to interact with your environment through natural-language conversations. When you describe your goals, the MCP server recommends the optimal model from your model registry by matching your intent against benchmarks such as MMLU and HumanEval, with cost comparisons. You can also manage data science projects, create workbenches, and monitor pipeline runs.

When you are ready to deploy, the MCP server generates three production-ready Kubernetes manifests for model serving, auto-scaling, and observability. The MCP server works with AI coding assistants such as Claude Code, OpenCode, and Gemini CLI. For more information, see Content from github.com is not included.RHOAI MCP server.

Red Hat OpenShift AI now displays tool callin, also known as function calling, configuration metadata directly in the model catalog. This update ensures that users have immediate access to the specific CLI arguments and chat templates required to successfully deploy models for agentic or tool-use workflows.

Previously, users had to manually identify which arguments were necessary for a model to interact with external APIs or functions. Validated models in the catalog now include this technical metadata as a single source of truth, accessible directly on the model card.

Key features of this update include:

The MCP Catalog provides a centralized experience for discovering, deploying, and experimenting with Model Context Protocol (MCP) servers in Red Hat OpenShift AI. AI Operators and Platform Engineers can browse available MCP servers in a catalog UI, view descriptive metadata about each server’s capabilities and tools, and deploy MCP servers into their namespace directly from the catalog. After deployment, a platform engineer can register deployed MCP servers in the gen AI studio configuration, making them available in the playground for interactive experimentation.

The MCP Catalog ships pre-loaded with MCP servers from Red Hat, technology partners, and the open source community. These servers can be deployed directly from the catalog without sourcing container images or configuring transport manually. Some additional prerequisite steps, such as mirroring images may be required for disconnected deployments or configuring server-specific credentials.

The MCP Catalog relies on the following pre-requisite upstream community components:

The Evaluation Stack control plane provides an API REST routing and orchestration layer for AI evaluation, benchmarking, and profiling backends on OpenShift AI. AI engineers can deploy and manage a comprehensive evaluation platform that supports multiple frameworks and execution modes through a unified interface.

The Evaluation Stack control plane includes the following capabilities:

The Kubeflow Spark Operator is now available in OpenShift AI as a Developer Preview. This feature provides Apache Spark integration with OpenShift AI, enabling the full lifecycle of Spark applications on Kubernetes. This enables unified orchestration and monitoring of large-scale data processing and preparation Spark jobs alongside ML training and inference workflows.

To enable the Kubeflow Spark Operator, navigate to your dsc.yaml CR and update the kubeflowsparkoperator parameter to the Managed state.

This feature introduces the following capabilities:

You can now run evaluations using LM-Eval on OGX with TrustyAI as a Developer Preview feature, using the built-in LM-Eval component and advanced content moderation tools. To use this feature, ensure TrustyAI is enabled, the FMS Orchestrator and detectors are set up, and KServe RawDeployment mode is in use for full compatibility if needed. There is no manual set up required.

Then, in the DataScienceCluster custom resource for the Red Hat OpenShift AI Operator, set the spec.components.ogx.managementState field to Managed.

For more information, see the following resources on GitHub:

LLM Compressor capabilities are now available in Red Hat OpenShift AI as a Developer Preview feature. A new workbench image with the llm-compressor library and a corresponding data science pipelines runtime image make it easier to compress and optimize your large language models (LLMs) for efficient deployment with vLLM. For more information, see Content from github.com is not included.llm-compressor in GitHub.

You can use LLM Compressor capabilities in two ways:

This feature is available as a Developer Preview.

You can now access and consume Model-as-a-Service (MaaS) models directly from the AI Available Assets page in the GenAI Studio.

Administrators can configure a MaaS by enabling the toggle in the Model Deployments page. When a model is marked as a service, it becomes global and visible across all projects in the cluster.

This feature is available as a Developer Preview.

Administrators can now add metadata to models during deployment so that they are automatically listed on the AI Available Assets page.

The following table describes the new metadata fields that streamline the process of making models discoverable and consumable by other teams:

This feature is available as a Developer Preview.

OGX remote providers and the SDK are now compatible with the Model Context Protocol (MCP) HTTP streaming protocol.

This enhancement enables developers to build fully stateless MCP servers, simplify deployment on standard OGX infrastructure (including serverless environments), and improve scalability. It also prepares for future enhancements such as connection resumption and provides a smooth transition away from Server-Sent Events (SSE).

This feature is available as a Developer Preview.

All Inference Time Scaling (ITS) runtime dependencies are now packaged in the Red Hat-maintained Python index, allowing Red Hat AI and OpenShift AI customers to install its_hub and its dependencies directly by using pip.

This enhancement enables users to build custom inference images with ITS algorithms focused on improving model accuracy at inference time without requiring model retraining, such as:

Static hardware profile support is now available to help users select training methods, models, and hyperparameters based on VRAM requirements and reference benchmarks. This approach ensures predictable and reliable training workflows without dynamic hardware discovery.

The following components are included:

Human-in-the-Loop (HIL) functionality has been added to the OGX agent to allow users to approve unread tool calls before execution.

This enhancement includes the following capabilities:

The tf2onnx package has been removed from TensorFlow workbench and runtime images. This package, which converts TensorFlow models to ONNX format, was incompatible with Keras 3 (used in TensorFlow 2.16+) and had irreconcilable dependency conflicts with protobuf versions required by onnx, tensorflow, and feast. The upstream project has been abandoned since January 2024.

If you require TensorFlow to ONNX conversion, see This content is not included.RHAIENG-1632 for alternative approaches.

The caikit-nlp component has been formally deprecated and removed from OpenShift AI 3.0.

This runtime is no longer included or supported in OpenShift AI. Users should migrate any dependent workloads to supported model serving runtimes.

The TGIS component, which was deprecated in OpenShift AI 2.19, has been removed in OpenShift AI 3.0.

TGIS continued to be supported through the OpenShift AI 2.16 Extended Update Support (EUS) lifecycle, which ended in June 2025.

Starting with this release, TGIS is no longer available or supported. Users should migrate their model serving workloads to supported runtimes such as Caikit or Caikit-TGIS.

The AppWrapper controller has been removed from OpenShift AI as part of the broader CodeFlare Operator removal process.

This change eliminates redundant functionality and reduces maintenance overhead and architectural complexity.

Download the updated pipeline definition for your experiment type from the rhoai-3.5-ea.1-fixed branch of the Content from github.com is not included.red-hat-data-services/pipelines-components repository on GitHub:

To prevent the service from failing, apply one of the following workarounds:

Cluster admins can create ClusterRoleBindings that apply the nemoguardrail-editor-role or nemoguardrail-viewer-role to users that require edit or view permissions to NeMo Guardrail resources:

$ oc create clusterrolebinding <binding-name> \
  --clusterrole=nemoguardrail-editor-role \
  --user=<username>

$ oc create clusterrolebinding <binding-name> \
  --clusterrole=nemoguardrail-viewer-role \
  --user=<username>

Delete the affected Deployment so that the operator recreates it with a valid strategy:

oc delete deployment <cr-name> -n <namespace>

Replace <cr-name> with the name of the OGXServer custom resource and <namespace> with its namespace. OGX operator will recreate deployment and new pod will work as expected.

Create a NetworkPolicy in the redhat-ods-applications namespace to allow communication between the pods created by the SparkApplication controller:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: spark-operator-allow-internal
spec:
  podSelector:
    matchLabels:
      sparkoperator.k8s.io/launched-by-spark-operator: "true"
  policyTypes:
    - Ingress
  ingress:
    - ports:
        - port: 7078
          protocol: TCP
        - port: 7079
          protocol: TCP
        - port: 4040
          protocol: TCP
      from:
        - podSelector: {}
        - namespaceSelector:
            matchLabels:
              network.openshift.io/policy-group: ingress

Restart the Feast Operator after DSC deployment completes:

$ kubectl rollout restart deployment/feast-operator-controller-manager -n redhat-ods-applications

Create a custom ServingRuntime by duplicating the vllm-spyre-s390x-runtime ServingRuntime and removing the command section from the container specification. Keep all other configuration, including environment variables, ports, and volume mounts, unchanged.

The following example shows only the affected section. Your complete ServingRuntime must include all other fields from the original template:

apiVersion: serving.kserve.io/v1alpha1
kind: ServingRuntime
metadata:
  name: vllm-spyre-s390x-runtime-copy
spec:
  containers:
    - name: kserve-container
      image: <image>
      # Remove the 'command' section that appears here in the original
      args:
        - --model=/mnt/models
        - --port=8000
        - --served-model-name={{.Name}}
      # ... keep all env, ports, volumeMounts from original ...

Create a basic tier using the CLI, then configure its settings through the dashboard. You must have cluster administrator privileges for your OpenShift cluster to perform these steps:

Use the automatic MLflow workbench integration, which configures service account token authentication through the MLFLOW_TRACKING_AUTH environment variable. Annotate your workbench notebook with opendatahub.io/mlflow-instance and restart the workbench. For more information, see Enable MLflow integration for a workbench.

If you are not using a workbench, create an OpenShift Route directly to the MLflow service endpoints and use the Route URL as the MLFLOW_TRACKING_URI when you authenticate.