Prompt optimization

Before starting this tutorial, ensure that you:
- Understand the basics of Generative AI workflows in SAP AI Core.
- Are familiar with creating and managing prompt templates, artifacts, and object stores
- Have the required roles such as genai_manager or custom_evaluation.
- Have completed the Quick Start tutorial or equivalent setup for SAP AI Core and AI Launchpad access.

• Prompt Optimization in SAP AI Core connects the Prompt Registry, Object Store, and ML Tracking Service to form an end-to-end optimization workflow.
• The dataset (for example, Test-Data.json) is stored in the Object Store and registered as an artifact.
• During execution, the system uses the selected prompt template, metric, and model to evaluate multiple prompt variants.
• Metrics are tracked in the ML Tracking Service, and both the optimized prompt and results are saved back to the registry and object store.
• This process runs as an execution and is model-specific, ensuring the optimized prompt aligns with the target model’s behavior.

For hands-on execution and end-to-end reference, use the accompanying Prompt Optimization Notebook. It includes complete Python code examples that align with each step of this tutorial — from dataset preparation and artifact registration to configuration creation, execution, and result retrieval.

💡 Even though this tutorial provides stepwise code snippets for clarity, the notebook contains all required imports, object initializations, and helper functions to run the flow seamlessly in one place.

To use the notebook:
- Download and open notebook in your preferred environment (e.g., VS Code, JupyterLab).
- Configure your environment variables such as AICORE_BASE_URL, AICORE_AUTH_TOKEN, and object store credentials .
- Execute each cell in order to reproduce the complete prompt optimization workflow demonstrated in this tutorial.

The object store is used by Prompt Optimization to read datasets and store generated artifacts and results.
In most environments, a default object store is already registered.
If your workspace already shows an entry named default under Object Stores, you can skip this step.
Otherwise, follow the instructions below to register a new one.

The dataset provides the examples used by the Prompt Optimization process to evaluate and refine your input prompt.
Each record should contain a sample input message and its corresponding expected structured JSON output, which represents the correct behavior you want the model to learn.

Dataset structure

Each record must include:

- input – the user message or text prompt

- answer – the expected model response (in valid JSON format)

Example record from facility-train.json:

[
    {
        "fields": {
            "input": "Subject: Urgent Assistance Required for Specialized Cleaning Services\n\nDear ProCare 
			Facility Solutions Support Team. Could you please arrange for a specialized cleaning team to 
			visit our home at the earliest convenience? We would greatly appreciate it if this could be 
			prioritized since we want to host a large party this week.\n\nThank you for your prompt 
			attention to this matter. We look forward to your swift response and assistance.\n\nBest 
			regards,\n[Sender]"
        },
        "answer": "{\"categories\": {\"routine_maintenance_requests\": false, 
		\"customer_feedback_and_complaints\": false, \"training_and_support_requests\": false, 
		\"quality_and_safety_concerns\": false, \"sustainability_and_environmental_practices\": false, 
		\"cleaning_services_scheduling\": false, \"specialized_cleaning_services\": true, 
		\"emergency_repair_services\": false, \"facility_management_issues\": false, 
		\"general_inquiries\": false}, \"sentiment\": \"neutral\", \"urgency\": \"high\"}"
    },
  {...}
]

Guidelines

• Verify that all answer values are valid JSON strings following the schema defined in your prompt.

• Include diverse examples that represent various urgencies, sentiments, and categories.

• Save the file as facility-train.json.

• Ensure it’s available locally for upload in the next step.

The dataset used for optimization must be registered as an artifact in SAP AI Core.
Artifacts act as the link between your files stored in the object store and the services that use them during prompt optimization runs.Each artifact is uniquely identified by its name and associated with a scenario.

In this step, you’ll create an artifact entry for your prepared dataset (facility-train.json).

The optimization configuration defines how prompt optimization runs — it links the dataset artifact, prompt template, model, and metric into one executable setup.
When you run the optimization, SAP AI Core uses this configuration to iteratively tune your prompt so that the chosen metric (for example, json_exact_match) is maximized.

⚠️ Note: Model availability and versions (for example, gpt-4o:2024-08-06, gemini-2.5-pro:latest) may vary across SAP AI Core tenants. Always verify available models in Generative AI Hub → Models before use.
For the latest updates, refer to SAP Note 3437766 – Model Availability and Support for Generative AI Hub
.

After registering the optimization configuration, the next step is to execute the optimization run.
This execution launches the prompt optimization workflow in SAP AI Core, which iteratively refines your prompt using the specified dataset and metric.
When the execution completes, the optimized prompt and results will be stored automatically in the prompt registry and object store.

After triggering the prompt optimization execution, you can monitor the progress and verify its status in real time.
Monitoring helps ensure that your run completes successfully and allows you to access intermediate and final optimization results.

Once the prompt optimization execution completes successfully, the system generates an optimized version of your prompt and stores it in the Prompt Registry.
You can review the optimization results, inspect metrics, and compare the base and optimized prompts to understand how performance has improved.