Skip to Content

Prompt optimization

Beginner
45 min.
SAP Business Technology Platform, Beginner, Tutorial, Machine Learning, Artificial Intelligence
This tutorial demonstrates how to use Prompt Optimization in SAP AI Core to automatically refine prompt templates using labeled datasets and evaluation metrics.The process optimizes a prompt for a specific model, stores metrics in the ML Tracking Service, and saves the optimized prompt and results back to the Prompt Registry and Object Store.
You will learn
  • How to prepare datasets and object stores for prompt optimization.
I321506Smita NaikNovember 26, 2025
Created by
I321506
November 26, 2025
Contributors
I321506

Prerequisites

  1. BTP Account
    Set up your SAP Business Technology Platform (BTP) account.
    Create a BTP Account
  2. For SAP Developers or Employees
    Internal SAP stakeholders should refer to the following documentation: How to create BTP Account For Internal SAP Employee, SAP AI Core Internal Documentation
  3. For External Developers, Customers, or Partners
    Follow this tutorial to set up your environment and entitlements: External Developer Setup Tutorial, SAP AI Core External Documentation
  4. Create BTP Instance and Service Key for SAP AI Core
    Follow the steps to create an instance and generate a service key for SAP AI Core:
    Create Service Key and Instance
  5. AI Core Setup Guide
    Step-by-step guide to set up and get started with SAP AI Core:
    AI Core Setup Tutorial
  6. An Extended SAP AI Core service plan is required, as the Generative AI Hub is not available in the Free or Standard tiers. For more details, refer to
    SAP AI Core Service Plans

  • How to create and register prompt templates in the Prompt Registry.

  • How to configure and run prompt optimization via AI Launchpad, Bruno, and the Python SDK.

  • How to monitor executions, review metrics, and save optimized prompts for reuse.

Pre-Read

Before starting this tutorial, ensure that you:

  • Understand the basics of Generative AI workflows in SAP AI Core.

  • Are familiar with creating and managing prompt templates, artifacts, and object stores

  • Have the required roles such as genai_manager or custom_evaluation.

  • Have completed the Quick Start tutorial or equivalent setup for SAP AI Core and AI Launchpad access.

Architecture Overview

  • Prompt Optimization in SAP AI Core connects the Prompt Registry, Object Store, and ML Tracking Service to form an end-to-end optimization workflow.

  • The dataset (for example, Test-Data.json) is stored in the Object Store and registered as an artifact.

  • During execution, the system uses the selected prompt template, metric, and model to evaluate multiple prompt variants.

  • Metrics are tracked in the ML Tracking Service, and both the optimized prompt and results are saved back to the registry and object store.

  • This process runs as an execution and is model-specific, ensuring the optimized prompt aligns with the target model’s behavior.
    img

Notebook Reference

For hands-on execution and end-to-end reference, use the accompanying Prompt Optimization Notebook. It includes complete Python code examples that align with each step of this tutorial — from dataset preparation and artifact registration to configuration creation, execution, and result retrieval.

💡 Even though this tutorial provides stepwise code snippets for clarity,
the notebook contains all required imports, object initializations, and helper functions to run the flow seamlessly in one place.

To use the notebook:

  • Download and open notebook in your preferred environment (e.g., VS Code, JupyterLab).

  • Configure your environment variables such as AICORE_BASE_URL, AICORE_AUTH_TOKEN, and object store credentials .

  • Execute each cell in order to reproduce the complete prompt optimization workflow demonstrated in this tutorial.

  • Step 1
  • Step 2

    The object store is used by Prompt Optimization to read datasets and store generated artifacts and results.
    In most environments, a default object store is already registered.
    If your workspace already shows an entry named default under Object Stores, you can skip this step.
    Otherwise, follow the instructions below to register a new one.

  • Step 3

    The dataset provides the examples used by the Prompt Optimization process to evaluate and refine your input prompt.
    Each record should contain a sample input message and its corresponding expected structured JSON output, which represents the correct behavior you want the model to learn.

    Dataset structure

    Each record must include:

    • input – the user message or text prompt

    • answer – the expected model response (in valid JSON format)

    Example record from facility-train.json:

    json
    Copy
    [
    {
        "fields": {
            "input": "Subject: Urgent Assistance Required for Specialized Cleaning Services\n\nDear ProCare 
            Facility Solutions Support Team. Could you please arrange for a specialized cleaning team to 
            visit our home at the earliest convenience? We would greatly appreciate it if this could be 
            prioritized since we want to host a large party this week.\n\nThank you for your prompt 
            attention to this matter. We look forward to your swift response and assistance.\n\nBest 
            regards,\n[Sender]"
        },
        "answer": "{\"categories\": {\"routine_maintenance_requests\": false, 
        \"customer_feedback_and_complaints\": false, \"training_and_support_requests\": false, 
        \"quality_and_safety_concerns\": false, \"sustainability_and_environmental_practices\": false, 
        \"cleaning_services_scheduling\": false, \"specialized_cleaning_services\": true, 
        \"emergency_repair_services\": false, \"facility_management_issues\": false, 
        \"general_inquiries\": false}, \"sentiment\": \"neutral\", \"urgency\": \"high\"}"
    },
  {...}
]

    Guidelines

    • Verify that all answer values are valid JSON strings following the schema defined in your prompt.

    • Include diverse examples that represent various urgencies, sentiments, and categories.

    • Save the file as facility-train.json.

    • Ensure it’s available locally for upload in the next step.

  • Step 4

    The dataset used for optimization must be registered as an artifact in SAP AI Core.
    Artifacts act as the link between your files stored in the object store and the services that use them during prompt optimization runs.Each artifact is uniquely identified by its name and associated with a scenario.

    In this step, you’ll create an artifact entry for your prepared dataset (facility-train.json).

  • Step 5
  • Step 6

    The optimization configuration defines how prompt optimization runs — it links the dataset artifact, prompt template, model, and metric into one executable setup.
    When you run the optimization, SAP AI Core uses this configuration to iteratively tune your prompt so that the chosen metric (for example, json_exact_match) is maximized.

    ⚠️ Note: Model availability and versions (for example, gpt-4o:2024-08-06, gemini-2.5-pro:latest) may vary across SAP AI Core tenants. Always verify available models in Generative AI Hub → Models before use.
    For the latest updates, refer to SAP Note 3437766 – Model Availability and Support for Generative AI Hub
    .

  • Step 7

    After registering the optimization configuration, the next step is to execute the optimization run.
    This execution launches the prompt optimization workflow in SAP AI Core, which iteratively refines your prompt using the specified dataset and metric.
    When the execution completes, the optimized prompt and results will be stored automatically in the prompt registry and object store.

  • Step 8

    After triggering the prompt optimization execution, you can monitor the progress and verify its status in real time.
    Monitoring helps ensure that your run completes successfully and allows you to access intermediate and final optimization results.

  • Step 9

    Once the prompt optimization execution completes successfully, the system generates an optimized version of your prompt and stores it in the Prompt Registry.
    You can review the optimization results, inspect metrics, and compare the base and optimized prompts to understand how performance has improved.

Back to Top
Back to top