Skip to Content

Consumption of GenAI models Using Orchestration - A Beginner's Guide

Beginner
45 min.
SAP AI Core, Beginner, Tutorial, Machine Learning, Artificial Intelligence
In this tutorial, we are going to learn the simple consumption of Gen AI models using the Orchestration.
You will learn
  • How to inference GenAI models Using Orchestration
I321506Smita NaikJuly 10, 2025
Created by
sharmaneeleshsap
November 25, 2024
Contributors
sharmaneeleshsap
I321506
TillK17

Prerequisites

  1. BTP Account
    Set up your SAP Business Technology Platform (BTP) account.
    Create a BTP Account
  2. For SAP Developers or Employees
    Internal SAP stakeholders should refer to the following documentation: How to create BTP Account For Internal SAP Employee, SAP AI Core Internal Documentation
  3. For External Developers, Customers, or Partners
    Follow this tutorial to set up your environment and entitlements: External Developer Setup Tutorial, SAP AI Core External Documentation
  4. Create BTP Instance and Service Key for SAP AI Core
    Follow the steps to create an instance and generate a service key for SAP AI Core:
    Create Service Key and Instance
  5. AI Core Setup Guide
    Step-by-step guide to set up and get started with SAP AI Core:
    AI Core Setup Tutorial
  6. An Extended SAP AI Core service plan is required, as the Generative AI Hub is not available in the Free or Standard tiers. For more details, refer to
    SAP AI Core Service Plans
  • Step 1

    This tutorial provides a basic introduction to using orchestration in SAP AI Core.

    Orchestration in SAP AI Core is a managed service that enables unified access, control, and execution of generative AI models through standardized APIs, templating, and configurable AI workflow components.

    You will learn how to deploy and configure orchestration to enable the consumption of multiple GenAI models within a single workflow.

    We will walk through a step-by-step guide and demonstrate the orchestration flow using a resume processing use case. This real-world scenario highlights how different models can collaborate within a cohesive pipeline using orchestration.

    Note: In SAP AI Core, orchestration deployment is available by default in the default resource group during the onboarding. For any new or additional resource groups, you must deploy a separate orchestration setup.

    While orchestration in SAP AI Core offers capabilities such as data masking, content filtering, translation, and grounding, this tutorial focuses on the basic consumption flow using mandatory modules like templating and model configuration. Others modules are Optional and usage of those modules are covered in a separate tutorial.

    By the end of this tutorial,

    • you will have a foundational understanding of orchestration through its minimal usage, focusing on practical application of templates and how to switch between different models using harmonized APIs.

    • Learn how to implement the solution using SAP AI Launchpad, Python SDK, Java, JavaScript, and Bruno.

    Refer to the orchestration documentation for more information.

  • Step 2
  • Step 3

    Execute this step only if orchestration deployment is not available. Other wise skip this step and proceed to the next step Consume LLM's in Generative AI Hub through Orchestration

    As part of the SAP AI Core onboarding process, an orchestration deployment is automatically created in the default resource group.

    This means you can start using orchestration in the Generative AI Hub right away—no need to create a separate deployment.

    In this step you will be:

    • creating the configuration required for the orchestration deployment
    • Creating the Orchestration deployment
  • Step 4
Back to Top
Back to top