--- parser: v2 auto_validation: true time: 20 tags: [tutorial>beginner, topic>machine-learning, topic>artificial-intelligence, topic>cloud, software-product>sap-business-technology-platform, software-product>sap-ai-services, software-product>business-entity-recognition, tutorial>free-tier] primary_tag: topic>machine-learning author_name: Juliana Morais author_profile: https://github.com/Juliana-Morais slug: cp-aibus-ber-custom-data canonical_url: https://developers.sap.com/tutorials/cp-aibus-ber-custom-data --- # Create Dataset and Upload Training Data Perform the initial steps to train your own Business Entity Recognition custom model to get machine learning entity predictions for the text you submit to the service. ## You will learn - How to call and test Business Entity Recognition - How to access and use the Business Entity Recognition Swagger UI (User Interface) - How to create a dataset, upload training data and submit a training job ## Intro The core functionality of Business Entity Recognition is to automatically detect and highlight any given type of named entity in unstructured text and classify it in accordance with predefined categories. In this tutorial, you will learn how to use the service APIs to create a dataset, upload training data, and submit a training job to finally create your own custom machine learning model to classify any given type of named entity, such as mobile number, first name, last name and address, in the texts you will submit to the service in the next tutorial: [Deploy Model and Get Prediction Results](cp-aibus-ber-custom-deploy). Check out the tutorial group [Use Pre-Trained Machine Learning Models to Process Unstructured Text](group.cp-aibus-business-entity-detect) to try out the Business Entity Recognition pre-trained machine learning models. --- ### Authorize Swagger UI You will use Swagger UI, via any web browser, to call the Business Entity Recognition service APIs. Swagger UI allows developers to effortlessly interact and try out every single operation an API exposes for easy consumption. For more information, see [Swagger UI](https://swagger.io/tools/swagger-ui/). In the service key you created for Business Entity Recognition in the previous tutorial: [Create Service Instance for Business Entity Recognition](cp-aibus-ber-service-instance), you should find (outside the `uaa` section of the service key) an entry called `url` (as highlighted in the image below). 1. To access the Business Entity Recognition Swagger UI, add **`/api/v1`** to the `url` value, paste it into any web browser and press **Enter**. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/service-key-details.png) 2. To be able to use the Swagger UI endpoints, you need to authorize yourself. In the top right corner, click **Authorize**. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/swagger.png) 3. Get the `access_token` value created in the previous tutorial: [Get OAuth Access Token for Business Entity Recognition Using Any Web Browser](cp-aibus-ber-web-oauth-token), then add **bearer** in front of it, and enter in the **Value** field. ``` bearer ``` ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/Authorize.png) 4. Click **Authorize** and then click **Close**. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/Authorize2.png) ### Create dataset Use the **POST /datasets** endpoint to create a dataset that will be used to train a custom model to extract entities from unstructured text. 1. Click the endpoint name to expand it. 2. Click **Try it out**. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/post-datasets-1.png) 3. In **payload**, enter a `description` for your dataset, `"Tutorial dataset"`, for example. 4. Click **Execute**. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/post-datasets-2.png) 5. Copy the **`datasetId`** from the **Response body**. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/post-datasets-3.png) ### See dataset details To see the details of your newly created dataset, use the **GET /datasets/{`datasetId`}** endpoint. 1. Click the endpoint name to expand it. 2. Click **Try it out**. 3. Enter the **`datasetId`** obtained in the previous step and click **Execute**. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/get-datasets-1.png) You should receive a response like below. Among the `datasetId` and the `description`, you see the `documentCount`. The number of documents include the training data files that you'll upload in the next step. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/get-datasets-2.png) ### Upload training data To upload training documents to your dataset, use the **POST /datasets/{`datasetId`}/documents**. You may repeat this step to upload multiple documents. Please bear in mind that Business Entity Recognition requires your data to be in a specific format to train a model. See data format prerequisites in [Training Data Format](https://help.sap.com/viewer/b43f8f61368d455793a241d2b10baeb2/SHIP/en-US/a1df156894ed4df7810d0a5abcce8d92.html). >As an alternative to uploading your own JSON file to the service, you can use the following sample files (right click on the link, then click ***Save link as*** to download the files locally): >- [Sample Training Data 1](https://github.com/SAPDocuments/Tutorials/raw/master/tutorials/cp-aibus-ber-custom-data/Tutorial_training_data_1.json) >- [Sample Training Data 2](https://github.com/SAPDocuments/Tutorials/raw/master/tutorials/cp-aibus-ber-custom-data/Tutorial_training_data_2.json) >Please repeat this step twice and upload one document each time. The more data is available the better predictions the model can give you. To upload documents, do the following: 1. Click the **POST /datasets/{`datasetId`}/documents** endpoint name to expand it. 2. Click **Try it out**. 3. Enter the **`datasetId`** obtained in step 2 and select one training document for the upload. 4. Click **Execute**. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/post-datasets-docs-1.png) 5. Copy the **`documentId`** from the **Response body**. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/post-datasets-docs-2.png) ### See document details Using the `datasetId` obtained in the previous step, you can obtain the details of your training documents using the **GET /datasets/{`datasetId`}/documents/{`documentId`}** endpoint. 1. Click the endpoint name to expand it. 2. Click **Try it out**. 3. Enter the **`datasetId`** obtained in step 2. 4. Enter the **`documentId`** obtained in step 4. 5. Click **Execute**. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/get-datasets-docs-1.png) You should receive a response, with the document details, like below. This shows you the size of the document in bytes. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/get-datasets-docs-2.png) ### Submit training job Once the training documents are uploaded, you can submit a training job. This triggers the training of a machine learning model based on your uploaded data. For that, use the **POST /training/jobs** endpoint. 1. Click the endpoint name to expand it. 2. Click **Try it out**. 3. In **payload**, enter a **`modelName`** for your custom model, `"Tutorial_custom_model"`, for example, and the **`datasetId`** obtained in step 2. >**CAUTION:** >Be aware of the following **`modelName`** guidelines:​ >- You cannot use the same name as one of the Business Entity Recognition pre-trained models (see [Extracted Entities by Pre-trained Model](https://help.sap.com/docs/Business_Entity_Recognition/b43f8f61368d455793a241d2b10baeb2/7eb1408001564d679adcd3bc4796800f.html)) >- You cannot use `"sap_"` as a prefix >- Start with an alphanumeric character >- You can include the special characters `"-"` and `"_"` >- You can use a maximum of 64 characters 4. Click **Execute**. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/post-training-jobs-1.png) 5. Copy the **`jobId`** from the **Response body**. This allows you to check the status of the training. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/post-training-jobs-2.png) This indicates that your training job has been successfully submitted. ### See training job status To check whether your training already succeeded, you can use the **GET /training/jobs/{`jobId`}** endpoint to see the current status of the job. >**CAUTION:** >In the free tier version of the service, only one training job with `SUCCEEDED` status is allowed. See [Free Tier Option Technical Constraints](https://help.sap.com/docs/Business_Entity_Recognition/b43f8f61368d455793a241d2b10baeb2/c80a45cc7416409eb9bf64667487c375.html). Use the endpoint **DELETE /training/jobs/{`jobId`}** to delete a training job before its status changes to `SUCCEEDED`. 1. Click the endpoint name to expand it. 2. Click **Try it out**. 3. Enter the **`jobId`** obtained in step 6. 4. Click **Execute**. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/get-training-jobs-1.png) You should receive a response like below. The status `RUNNING` indicates that the training is still in progress. In case the status is `PENDING`, then the training has not started yet. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/get-training-jobs-2.png) You may check the status now and then. Please note that the training may take up to 5 hours. Afterwards, the training status changes to `SUCCEEDED`. Along with that, you receive all the capabilities of the model, that are the entities the model can recognize. ![BER](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/cp-aibus-ber-custom-data/get-training-jobs-3.png)